[x264-devel] [BUG] segfault in x264_weight_cost_init_luma

[Jason Garrett-Glaser]

On Sat, Oct 9, 2010 at 9:30 PM, Corey Hickey <bugfood-ml at fatooh.org> wrote:
> Hi,
>
> I am encountering a segfault under the following conditions:
> 1. using mencoder
> 2. with x264 set to use one thread
> 3. when the second pass has 2 more frames than the first
>
> x264 prints the following errors:
>
> x264 [error]: Incomplete MB-tree stats file.
> x264_encoder_encode failed
> x264 [error]: 2nd pass has more frames than 1st pass (1)
> x264 [error]: continuing anyway, at constant QP=24
> x264 [error]: disabling adaptive B-frames
>
>
> (full logs attached)
>
>
> Originally, it would appear to be mencoder's bug that it is feeding x264
> more frames, but I am having trouble debugging that bug because of this
> segfault. Rather than send in the large file with which I originally
> encountered the issue, I can make the segfault happen by manually
> specifying more frames on the second pass.
>
> I distilled the steps to reproduce down to a small script, which I have
> attached, along with some gdb info from the second pass.
>
> I did my best to reproduce this with the x264 program itself, but I
> haven't been able to. It seems that x264 detects that there are too many
> frames in the second pass and quits first.

Is mencoder passing frames to x264 after x264_encoder_encode fails?
You're really not supposed to do that.

Dark Shikari