[x264-devel] [Bug Report] Segmentation Fault in 2nd Pass

[Rodolfo J. Quesada Zumbado]

Quite odd indeed!

Among the tests I tried, I initialized the structure member 
analysis.i_predict8x8chroma to values out of the [0-7] range that would 
be out of bounds in the array access, causing the segfault.

And when setting it to something like 99999, it propagated to the array 
untouched.

I didn't check all the code that might update or change the 
analysis.i_predict8x8chroma values, only checking their values before 
and after, but maybe it isn't picking a mode under a certain weird 
condition.

Could it be a MEncoder issue? Something like, sending a bad frame to the 
analysis code or the like? Besides, it only occurs during a seconds pass.

Jason Garrett-Glaser wrote:
> On Sun, Jun 8, 2008 at 10:34 AM, Rodolfo J. Quesada Zumbado
> <rquesada at roqz.net> wrote:
>> Hi,
>>
>> I've playing with the x264 code to see what I could find about the
>> aforementioned bug, using the git head that now produces a
>> libx264.so.60.
>>
>> I traced the data flow from x264_rd_cost_i8x8_chroma() in rdo.c and I
>> found that the error further propagates to cabac.c in
>> x264_i8x8_chroma_size_cabac(), from there to
>> x264_cabac_mb_intra_chroma_pred_mode() and there the error occurs when
>> trying to access a value from the array x264_mb_pred_mode8x8c_fix[7]
>> with the mysterious index "1151172282", always the same value, and
>> clearly out of bounds for that array and that's why the segfault occurs.
> 
> This is very interesting, because I remember seeing this exact same
> issue occur before--but it *only occurred when running through
> mplayer*, which is just patently ridiculous and bizarre.  And of
> course as you mentioned, the fprintf eliminated the bug, which makes
> this even harder to debug (a "Heisenbug"!).
> 
> I'm basically 100% sure that intra_chroma analysis is guaranteed to
> pick some mode; I don't think its possible for it not to.  But this is
> really quite odd...
> 
> Dark Shikari
> _______________________________________________
> x264-devel mailing list
> x264-devel at videolan.org
> http://mailman.videolan.org/listinfo/x264-devel

-- 
Rodolfo J. Quesada Zumbado
rquesada at roqz.net