[x264-devel] Feedbacks - x264 benchmark

[BugMaster]

On Fri, 12 Feb 2021 17:38:27 +0100 (CET), Luc Lesoil wrote:
> Dear x264 developers, 

> We take the liberty of contacting you because we benchmarked x264
> on a large number of input videos (see the measurements here : [
> https://zenodo.org/record/3928253 |
> https://zenodo.org/record/3928253 ] ), and would like to benefit
> from your expertise as x264's developers. 

> To summarize, our goal is to configure x264 for its input video
> (like the configuration option tune ) so it decreases the bitrate of the encoding process.

> Our findings are summarized in the slides (<10 slides) available
> here : [
> https://filesender.renater.fr/?s=download&token=7796e4a5-42ba-44f4-b553-cb5daf7d907e
> | https://filesender.renater.fr/?s=download&to ] [
> mailto:luc.lesoil at irisa ] [
> https://filesender.renater.fr/?s=download&token=7796e4a5-42ba-44f4-b553-cb5daf7d907e
> | ken=7796e4a5-42ba-44f4-b553-cb5daf7d907e ] 

> If you have five minutes, we would interested to have you feedback on 3 points:

> 1. What do you think of the dataset of videos we used in this
> experiment? Is it a realistic benchmark to estimate the usage of
> x264? And what do you use in terms of input videos to benchmark x264?

> 2. Any advice to improve the experiment (like configuration options
> that could be missing in the experiment)? 

> 3. We found four groups of videos that have similar performances ,
> 1. actions videos, 2. big resolution videos, 3. Still image videos,
> 4. No specific property (or standard videos?). Does that correspond
> to something you already observed with x264? And how did you isolate
> the different categories related to the tune feature (see [
> https://superuser.com/questions/564402/explanation-of-x264-tune |
> https://superuser.com/questions/564402/explanation-of-x264-tune ] )? 

> You can send your feedbacks at this mail address. Or if it is too
> long to write, we can organize a visio conference. 

> In advance, thank you for your time! 

> Best regards, 
> Luc Lesoil 
> [ mailto:luc.lesoil at irisa | luc.lesoil at irisa.fr ] 
> PhD Student 
> Université de Rennes 1, Bretagne, France 

Hi.

1) In your slides page 3 "Motivation : why x264?" you have a lot of
factual errors with authors and dates of commits. [1]

2) Why do you use version from 2017 (x264 version: 0.152.2854, git
commit e9a5903) in comparison made in 2020?

3) I didn't understand what you was trying to measure. Your output
data doesn't have any quality metric but only size (bitrate) and time
(fps) for different configurations. So I am not sure what conclusion
you was going to have from it because such result are not comparable
(different bitrate and different quality).

[1] git commit logs:
commit 835ccc3cec908b1febfd31613d3e6583628116b3
Author: Fiona Glaser <fiona at x264.com>
Date:   Tue Aug 4 17:46:33 2009 -0700

    Macroblock-tree ratecontrol
    On by default; can be turned off with --no-mbtree.
    Uses a large lookahead to track temporal propagation of data and weight quality accordingly.
    Requires a very large separate statsfile (2 bytes per macroblock) in multi-pass mode.
    Doesn't work with b-pyramid yet.
    Note that MB-tree inherently measures quality different from the standard qcomp method, so bitrates produced by CRF may change somewhat.
    This makes the "medium" preset a bit slower.  Accordingly, make "fast" slower as well, and introduce a new preset "faster" between "fast" and "veryfast".
    All presets "fast" and above will have MB-tree on.
    Add a new option, --rc-lookahead, to control the distance MB tree looks ahead to perform propagation analysis.
    Default is 40; larger values will be slower and require more memory but give more accurate results.
    This value will be used in the future to control ratecontrol lookahead (VBV).
    Add a new option, --no-psy, to disable all psy optimizations that don't improve PSNR or SSIM.
    This disables psy-RD/trellis, but also other more subtle internal psy optimizations that can't be controlled directly via external parameters.
    Quality improvement from MB-tree is about 2-70% depending on content.
    Strength of MB-tree adjustments can be tweaked using qcompress; higher values mean lower MB-tree strength.
    Note that MB-tree may perform slightly suboptimally on fades; this will be fixed by weighted prediction, which is coming soon.

commit f30aed6d810ef408cbf19cc6760605b0b87cbfde
Author: Fiona Glaser <fiona at x264.com>
Date:   Fri Dec 11 17:22:18 2009 -0800

    Fix two bugs in 2-pass ratecontrol
    last_qscale_for wasn't set during the 2pass init code.
    abr_buffer was way too small in the case of multiple threads, so accordingly increase its buffer size based on the number of threads.
    May significantly increase quality with many threads in 2-pass mode, especially in cases with extremely large I-frames, such as anime.

commit 76a8276f19ca5b01b3d54858cfc95ddc20fb2a71
Author: Fiona Glaser <fiona at x264.com>
Date:   Sun Feb 21 03:56:06 2010 -0800

    Make b-pyramid normal the default
    Now that b-pyramid works with MB-tree and is spec compliant, there's no real reason not to make it default.
    Improves compression 0-5% depending on the video.
    Also allow 0/1/2 to be used as aliases for none/strict/normal (for conciseness).

commit 134e221530d246e78f986e14d1f6d25a52bb3836
Author: Fiona Glaser <fiona at x264.com>
Date:   Fri Apr 9 18:13:22 2010 -0700

    Add faster mv0 special case for macroblock-tree
    Improves performance on low-motion video.

commit b6b8aea6baaac8284a61f5879ba94a26a3cd6156
Author: Alex Wright <alexw0885 at gmail.com>
Date:   Sun Sep 19 05:08:22 2010 -0700

    Chroma mode decision/subpel for B-frames
    Improves compression ~0.4-1%. Helps more on videos with lots of chroma detail.
    Enabled at subme 9 (preset slower) and higher.

commit b3fb718404d6cce9c82987ea2909cda5072d040c
Author: Fiona Glaser <fiona at x264.com>
Date:   Sun Feb 23 10:36:55 2014 -0800

    Macroblock tree overhaul/optimization
    
    Move the second core part of macroblock tree into an assembly function;
    SIMD-optimize roughly half of it (for x86). Roughly ~25-65% faster mbtree,
    depending on content.
    
    Slightly change how mbtree handles the tradeoff between range and precision
    for propagation.
    
    Overall a slight (but mostly negligible) effect on SSIM and ~2% faster.