[x264-devel] [Bug Report] Segmentation Fault in 2nd Pass

[Jason Garrett-Glaser]

On Sun, Jun 8, 2008 at 10:34 AM, Rodolfo J. Quesada Zumbado
<rquesada at roqz.net> wrote:
> Hi,
>
> I've playing with the x264 code to see what I could find about the
> aforementioned bug, using the git head that now produces a
> libx264.so.60.
>
> I traced the data flow from x264_rd_cost_i8x8_chroma() in rdo.c and I
> found that the error further propagates to cabac.c in
> x264_i8x8_chroma_size_cabac(), from there to
> x264_cabac_mb_intra_chroma_pred_mode() and there the error occurs when
> trying to access a value from the array x264_mb_pred_mode8x8c_fix[7]
> with the mysterious index "1151172282", always the same value, and
> clearly out of bounds for that array and that's why the segfault occurs.

This is very interesting, because I remember seeing this exact same
issue occur before--but it *only occurred when running through
mplayer*, which is just patently ridiculous and bizarre.  And of
course as you mentioned, the fprintf eliminated the bug, which makes
this even harder to debug (a "Heisenbug"!).

I'm basically 100% sure that intra_chroma analysis is guaranteed to
pick some mode; I don't think its possible for it not to.  But this is
really quite odd...

Dark Shikari