[x264-devel] Feedbacks - x264 benchmark
Luc Lesoil
luc.lesoil at irisa.fr
Mon Feb 15 08:52:32 UTC 2021
"You should not only measure quality at some random bitrate but to make
them comparable you should measure it at same/equal bitrate (because
to make quality itself same/equal is much harder or mostly impossible
task). Or you can measure quality and bitrate at few points of
quality/bitrate curve and compare this RD curves (but this RD curves
should be overlapping to be comparable)."
Ok, we'll look at it.
"Which Fedora version are you using to get x264 from 2017? Fedora doesn't
ship x264 due to patent concerns, so I assume you meant RPM Fusion. The
latest version of Fedora that RPM Fusion shipped x264 from 2017 for was
Fedora 28 or 29, which are 3 years out of date.
RPM Fusion for Fedora 33 (latest release) ships
x264-0.160-2.20200702gitcde9a93.fc33.x86_64
Please be specific in your claims, because you won't get x264 from 2017
on current Fedora (33) when installing from official repositories
(Fedora + RPM Fusion).
Regards,
Dominik"
With Ubuntu 20.04 LTS, by installing x264 with the command line (i.e. typing 'sudo apt-get install x264'), you have a 2018 version.
'x264 --version'
x264 0.155.2917 0a84d98
(libswscale 5.1.100)
(libavformat 58.12.100)
(ffmpegsource 2.23.0.0)
built on Sep 27 2018, gcc: 8.2.0
x264 configuration: --chroma-format=all
libx264 configuration: --chroma-format=all
x264 license: GPL version 2 or later
libswscale/libavformat/ffmpegsource license: GPL version 2 or later
Same with debian
'cat /etc/debian_version'
10.4
'x264 --version'
x264 0.155.2917 0a84d98
(libswscale 5.1.100)
(libavformat 58.12.100)
(ffmpegsource 2.23.0.0)
built on Oct 3 2018, gcc: 8.2.0
x264 configuration: --chroma-format=all
libx264 configuration: --chroma-format=all
x264 license: GPL version 2 or later
libswscale/libavformat/ffmpegsource license: GPL version 2 or later
When we started these experiments (end 2019/beginning of 2020) it was the version mentioned in the slides.
----- Mail original -----
> De: "Luc Lesoil" <luc.lesoil at irisa.fr>
> À: "x264-devel" <x264-devel at videolan.org>
> Envoyé: Dimanche 14 Février 2021 14:55:25
> Objet: Re: Feedbacks - x264 benchmark
> Hi,
>
> 1) The messages are just excerpts of the real commit messages, we will correct
> that.
>
> 2) It's the version used when installing x264 with a command line on a linux
> distribution (raspbian, fedora, etc.), so mostly downloaded by users. You are
> right though, it would be nice to replicate this on the last commit of x264.
>
> 3) Ok, we can try to measure the quality (SSIM maybe?) and find a compromise
> between quality and bitrate.
>
> Thank you for your feedbacks!
>
> ----- Mail original -----
>> De: x264-devel-request at videolan.org
>> À: "x264-devel" <x264-devel at videolan.org>
>> Envoyé: Dimanche 14 Février 2021 13:00:01
>> Objet: x264-devel Digest, Vol 162, Issue 8
>
>> Send x264-devel mailing list submissions to
>> x264-devel at videolan.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> https://mailman.videolan.org/listinfo/x264-devel
>> or, via email, send a message with subject or body 'help' to
>> x264-devel-request at videolan.org
>>
>> You can reach the person managing the list at
>> x264-devel-owner at videolan.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of x264-devel digest..."
>>
>>
>> Today's Topics:
>>
>> 1. Re: Feedbacks - x264 benchmark (BugMaster)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Sat, 13 Feb 2021 22:55:50 +0300
>> From: BugMaster <BugMaster at narod.ru>
>> To: Mailing list for x264 developers <x264-devel at videolan.org>
>> Subject: Re: [x264-devel] Feedbacks - x264 benchmark
>> Message-ID: <1022719855.20210213225550 at narod.ru>
>> Content-Type: text/plain; charset=iso-8859-1
>>
>> On Fri, 12 Feb 2021 17:38:27 +0100 (CET), Luc Lesoil wrote:
>>> Dear x264 developers,
>>
>>> We take the liberty of contacting you because we benchmarked x264
>>> on a large number of input videos (see the measurements here : [
>>> https://zenodo.org/record/3928253 |
>>> https://zenodo.org/record/3928253 ] ), and would like to benefit
>>> from your expertise as x264's developers.
>>
>>> To summarize, our goal is to configure x264 for its input video
>>> (like the configuration option tune ) so it decreases the bitrate of the
>>> encoding process.
>>
>>> Our findings are summarized in the slides (<10 slides) available
>>> here : [
>>> https://filesender.renater.fr/?s=download&token=7796e4a5-42ba-44f4-b553-cb5daf7d907e
>>> | https://filesender.renater.fr/?s=download&to ] [
>>> mailto:luc.lesoil at irisa ] [
>>> https://filesender.renater.fr/?s=download&token=7796e4a5-42ba-44f4-b553-cb5daf7d907e
>>> | ken=7796e4a5-42ba-44f4-b553-cb5daf7d907e ]
>>
>>> If you have five minutes, we would interested to have you feedback on 3 points:
>>
>>> 1. What do you think of the dataset of videos we used in this
>>> experiment? Is it a realistic benchmark to estimate the usage of
>>> x264? And what do you use in terms of input videos to benchmark x264?
>>
>>> 2. Any advice to improve the experiment (like configuration options
>>> that could be missing in the experiment)?
>>
>>> 3. We found four groups of videos that have similar performances ,
>>> 1. actions videos, 2. big resolution videos, 3. Still image videos,
>>> 4. No specific property (or standard videos?). Does that correspond
>>> to something you already observed with x264? And how did you isolate
>>> the different categories related to the tune feature (see [
>>> https://superuser.com/questions/564402/explanation-of-x264-tune |
>>> https://superuser.com/questions/564402/explanation-of-x264-tune ] )?
>>
>>> You can send your feedbacks at this mail address. Or if it is too
>>> long to write, we can organize a visio conference.
>>
>>> In advance, thank you for your time!
>>
>>> Best regards,
>>> Luc Lesoil
>>> [ mailto:luc.lesoil at irisa | luc.lesoil at irisa.fr ]
>>> PhD Student
>>> Université de Rennes 1, Bretagne, France
>>
>> Hi.
>>
>> 1) In your slides page 3 "Motivation : why x264?" you have a lot of
>> factual errors with authors and dates of commits. [1]
>>
>> 2) Why do you use version from 2017 (x264 version: 0.152.2854, git
>> commit e9a5903) in comparison made in 2020?
>>
>> 3) I didn't understand what you was trying to measure. Your output
>> data doesn't have any quality metric but only size (bitrate) and time
>> (fps) for different configurations. So I am not sure what conclusion
>> you was going to have from it because such result are not comparable
>> (different bitrate and different quality).
>>
>> [1] git commit logs:
>> commit 835ccc3cec908b1febfd31613d3e6583628116b3
>> Author: Fiona Glaser <fiona at x264.com>
>> Date: Tue Aug 4 17:46:33 2009 -0700
>>
>> Macroblock-tree ratecontrol
>> On by default; can be turned off with --no-mbtree.
>> Uses a large lookahead to track temporal propagation of data and weight quality
>> accordingly.
>> Requires a very large separate statsfile (2 bytes per macroblock) in multi-pass
>> mode.
>> Doesn't work with b-pyramid yet.
>> Note that MB-tree inherently measures quality different from the standard qcomp
>> method, so bitrates produced by CRF may change somewhat.
>> This makes the "medium" preset a bit slower. Accordingly, make "fast" slower as
>> well, and introduce a new preset "faster" between "fast" and "veryfast".
>> All presets "fast" and above will have MB-tree on.
>> Add a new option, --rc-lookahead, to control the distance MB tree looks ahead to
>> perform propagation analysis.
>> Default is 40; larger values will be slower and require more memory but give
>> more accurate results.
>> This value will be used in the future to control ratecontrol lookahead (VBV).
>> Add a new option, --no-psy, to disable all psy optimizations that don't improve
>> PSNR or SSIM.
>> This disables psy-RD/trellis, but also other more subtle internal psy
>> optimizations that can't be controlled directly via external parameters.
>> Quality improvement from MB-tree is about 2-70% depending on content.
>> Strength of MB-tree adjustments can be tweaked using qcompress; higher values
>> mean lower MB-tree strength.
>> Note that MB-tree may perform slightly suboptimally on fades; this will be fixed
>> by weighted prediction, which is coming soon.
>>
>> commit f30aed6d810ef408cbf19cc6760605b0b87cbfde
>> Author: Fiona Glaser <fiona at x264.com>
>> Date: Fri Dec 11 17:22:18 2009 -0800
>>
>> Fix two bugs in 2-pass ratecontrol
>> last_qscale_for wasn't set during the 2pass init code.
>> abr_buffer was way too small in the case of multiple threads, so accordingly
>> increase its buffer size based on the number of threads.
>> May significantly increase quality with many threads in 2-pass mode, especially
>> in cases with extremely large I-frames, such as anime.
>>
>> commit 76a8276f19ca5b01b3d54858cfc95ddc20fb2a71
>> Author: Fiona Glaser <fiona at x264.com>
>> Date: Sun Feb 21 03:56:06 2010 -0800
>>
>> Make b-pyramid normal the default
>> Now that b-pyramid works with MB-tree and is spec compliant, there's no real
>> reason not to make it default.
>> Improves compression 0-5% depending on the video.
>> Also allow 0/1/2 to be used as aliases for none/strict/normal (for conciseness).
>>
>> commit 134e221530d246e78f986e14d1f6d25a52bb3836
>> Author: Fiona Glaser <fiona at x264.com>
>> Date: Fri Apr 9 18:13:22 2010 -0700
>>
>> Add faster mv0 special case for macroblock-tree
>> Improves performance on low-motion video.
>>
>> commit b6b8aea6baaac8284a61f5879ba94a26a3cd6156
>> Author: Alex Wright <alexw0885 at gmail.com>
>> Date: Sun Sep 19 05:08:22 2010 -0700
>>
>> Chroma mode decision/subpel for B-frames
>> Improves compression ~0.4-1%. Helps more on videos with lots of chroma detail.
>> Enabled at subme 9 (preset slower) and higher.
>>
>> commit b3fb718404d6cce9c82987ea2909cda5072d040c
>> Author: Fiona Glaser <fiona at x264.com>
>> Date: Sun Feb 23 10:36:55 2014 -0800
>>
>> Macroblock tree overhaul/optimization
>>
>> Move the second core part of macroblock tree into an assembly function;
>> SIMD-optimize roughly half of it (for x86). Roughly ~25-65% faster mbtree,
>> depending on content.
>>
>> Slightly change how mbtree handles the tradeoff between range and precision
>> for propagation.
>>
>> Overall a slight (but mostly negligible) effect on SSIM and ~2% faster.
>>
>>
>>
>> ------------------------------
>>
>> Subject: Digest Footer
>>
>> _______________________________________________
>> x264-devel mailing list
>> x264-devel at videolan.org
>> https://mailman.videolan.org/listinfo/x264-devel
>>
>>
>> ------------------------------
>>
>> End of x264-devel Digest, Vol 162, Issue 8
> > ******************************************
More information about the x264-devel
mailing list