[x264-devel] VBV Rate Control Challenge

[Jason Garrett-Glaser]

On Tue, Dec 29, 2009 at 5:55 PM, Mike Kazmier <DaKaZ at zenbe.com> wrote:
>
>
> On Tue, Dec 29, 2009 at 3:33 PM "Jason Garrett-Glaser" <darkshikari at gmail.com> wrote:
>>On Tue, Dec 29, 2009 at 5:07 PM, Mike Kazmier <DaKaZ at zenbe.com> wrote:
>> > All,
>> >
>> > I have an interesting test clip which consistently breaks VBV rate control.  In particular, the breakage is worse when using --rc-lookahead.  I find this interesting because I thought lookahead would help with this situation.  The scene is not necessarily one you would find in the wild (in other words, we were setup for failure) but none the less, I would expect x264 to handle it somewhat gracefully.  I am not sure of a strategy to do so beyond improving the heuristics of the lookahead to identify the amazing complex scene (static) that is coming and empty the VBV to the most extent possible while also cranking the QP all the way up.  Any thoughts are appreciated.  You can recreate the vbv underflow with this command:
>> >
>> > ffmpeg -i ./doremi-2.ts -f rawvideo -s 1920x1080 -| x264 -o ./doremi-2.h264 --threads 9 --trellis 1 --nr 1000 --bitrate 6500 --vbv-maxrate 6500 --vbv-bufsize 6000 --fps 30000/1001 --keyint 60 --min-keyint 15 --rc-lookahead 0 --interlaced --bframes 3 --b-adapt 1  - 1920x1080
>> >
>> > The sample clip is available here:
>> > http://www.mediafire.com/?hljzzzdm2ny
>> >
>> > Thanks,
>> >
>> > --DaKaZ
>>
>> This clip is impossible.
>>
>> Nothing to do with emptying VBV or lookahead or threads: it is simply
>> impossible to fit frames 259 and 260, via _any method_, even at QP51,
>> into the VBV buffer you have specified.  The only way to fix this
>> would be a VBV emergency mode of some sort, in which we force
>> skips/empty intra blocks or otherwise reduce coefficients even further
>> beyond max quantizer.
>>
>
> Yeah - I was trying to think of creative way to address this, I recognize its basically impossible and unrealistic, but it is a corner case and would be nice to handle it gracefully.  You have an interesting idea, which would be to: A) identify the amazing complex frame, and B) invoke some type of clipping, smoothing, blurring, etc of each MB in order to fit the frame into its VBV allocation.

Oh, there are a few options I've considered over time.  Feel free to
vote or suggest another.

1.  High-Complexity Emergency Mode
Add quantizers above 51.  These aren't "real" quantizers, of course:
they just map to an activation of a denoiser that lowers all DCT
coefficients in the residual by some factor.  The maximum quantizer,
say, 71, forces all blocks to skip (inter) and empty-CBP (intra).

2.  Low-complexity Emergency Mode
Add quantizer 52, which forces all blocks to skip or empty-CBP.

3.  Row-reencoding Emergency Mode
In VBV mode, cache the bitstream state at the start of the row and, in
the case of a massive misprediction, re-encode the row.  In the case
of underflow, re-encode the row as skips or empty-CBP blocks.

4.  Frame-dropping Emergency Mode.
Just drop frames if they underflowed.

> I don't have the tools handy to look, but what size buffer does x264 give frames 259 and 260?

Nothing to do with the size buffer it gives: everything shoots up to
QP51.  There is _nothing_ x264 can do beyond there.

>But... Even if you take your VBV bufsize up to, say, 9000.... we still have underflows.  One should reason that with the 50% larger buffer and... perhaps better lookahead vbv allocation, we could survive?

Buffer fill rate is too low too.  The sequence with random-bits
background simply has too high a residual to be absorbed by any
sane-sized buffer at that bitrate, from testing.

Jason