[x264-devel] [BUG] segfault in x264_weight_cost_init_luma

[Corey Hickey]

Hi,

I am encountering a segfault under the following conditions:
1. using mencoder
2. with x264 set to use one thread
3. when the second pass has 2 more frames than the first

x264 prints the following errors:

x264 [error]: Incomplete MB-tree stats file.
x264_encoder_encode failed
x264 [error]: 2nd pass has more frames than 1st pass (1)
x264 [error]: continuing anyway, at constant QP=24
x264 [error]: disabling adaptive B-frames


(full logs attached)


Originally, it would appear to be mencoder's bug that it is feeding x264
more frames, but I am having trouble debugging that bug because of this
segfault. Rather than send in the large file with which I originally
encountered the issue, I can make the segfault happen by manually
specifying more frames on the second pass.

I distilled the steps to reproduce down to a small script, which I have
attached, along with some gdb info from the second pass.

I did my best to reproduce this with the x264 program itself, but I
haven't been able to. It seems that x264 detects that there are too many
frames in the second pass and quits first.

mencoder: current svn
x264: current git
OS: Debian Sid, 64-bit
kernel: 2.6.35.5

Please let me know if I need to provide any more information.

Thanks,
Corey
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bad.sh
Type: application/x-sh
Size: 330 bytes
Desc: not available
URL: <http://mailman.videolan.org/pipermail/x264-devel/attachments/20101009/69794a5b/attachment.sh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pass1.log
URL: <http://mailman.videolan.org/pipermail/x264-devel/attachments/20101009/69794a5b/attachment.asc>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pass3.log
URL: <http://mailman.videolan.org/pipermail/x264-devel/attachments/20101009/69794a5b/attachment.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: gdb.log
URL: <http://mailman.videolan.org/pipermail/x264-devel/attachments/20101009/69794a5b/attachment-0001.asc>