[x264-devel] Re: Grey "fogged" frames in h264 encoded videos. Is this a bug?

[Loren Merritt]

On Thu, 21 Jun 2007, gregor koseleck wrote:

> I am trying to encode different movies to h264 using ffmpeg and x264.
> The system where the encoding happens, is a 64bit Gentoo linux (stage
> 3).
> I tried different parameters of ffmpeg but can not get rid of the grev
> fog in black areas of the videos. The problem appears only on some
> frames and lasts different times. It seems that it depends on the GOP
> setting: When I set GOP to high (300 or more), the problem apears only
> some times but always on the same input files and the same time.
> ffmpeg shows an output of the library telling something about an
> overflow.

When encoding CAVLC in Baseline or Main profile, there is a limit on the 
size of any one DCT coefficient. This limit can only be reached with very 
low QP or with extreme quantization matrices, so it shouldn't normally be 
a problem, but ffmpeg has lousy defaults (such as min QP=2) which allow 
such extreme cases.
And even though larger values are allowed in High profile (it needs them 
to represent higher sample depth), I haven't implemented that in x264.
There is no direct relation between GOP size and overflow, though I-frames 
are lower QP than P-frames and I-blocks are more likely to contain large 
coefficients than P-blocks.

(pick any one of the following)
What you can do about it without modifying any source code:
Enable CABAC.
Set min QP to something reasonable (like x264's default 10).

What you can do about it with modifying source code:
Add codec-specific defaults in ffmpeg so that h264 doesn't default to 
insane settings.
Make x264 encode that coefficient using High profile's larger level_code, 
resulting in a syntactically valid stream but violating profile 
constraints.
Make x264 re-encode the macroblock with a higher QP.
Make x264 re-decode the macroblock so that the reconstructed picture 
matches the bitstream; there will still be an artifact but it will be 
localized to a few blocks and fixed in the next frame.

--Loren Merritt

-- 
This is the x264-devel mailing-list
To unsubscribe, go to: http://developers.videolan.org/lists.html