[x264-devel] segfault in x264_mb_encode_chroma_internal

[Jason Garrett-Glaser]

On Tue, Jun 19, 2012 at 2:44 PM, Miklos Juhasz
<juhasz.miklos.1 at gmail.com> wrote:
> Hi,
>
> I've been encoding with Handbrake using the latest version of x264
> (master, last commit 2012-05-18) on Ubuntu Linux 12.04 and experienced
> quite frequently that the program stopped with general protection error.
> Once it made X segfault and once even caused kernel panic. I am running
> an i7 2600K, 12 threads.
>
> Using debug symbols and gdb here's the backtrace:
>
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff917fa700 (LWP 4401)]
> 0x0000000000c96d09 in x264_mb_encode_chroma_internal (chroma422=0, i_qp=21,
>    b_inter=0, h=0x7fffea255d60) at encoder/macroblock.c:367
>
> #0  0x0000000000c96d09 in x264_mb_encode_chroma_internal (chroma422=0,
>    i_qp=21, b_inter=0, h=0x7fffea255d60) at encoder/macroblock.c:367
> #1  x264_mb_encode_chroma (h=0x7fffea255d60, b_inter=0, i_qp=21)
>    at encoder/macroblock.c:457
> #2  0x0000000000c98636 in x264_macroblock_encode_internal (chroma=1,
>    plane_count=1, h=0x7fffea255d60) at encoder/macroblock.c:888
> #3  x264_macroblock_encode (h=0x7fffea255d60) at encoder/macroblock.c:927
> #4  0x0000000000c62a1e in x264_rd_cost_mb (h=0x7fffea255d60, i_lambda2=3686)
>    at encoder/rdo.c:162
> #5  0x0000000000c696aa in x264_intra_rd (h=0x7fffea255d60,
> a=0x7fff917f4180,
>    i_satd_thresh=3162) at encoder/analyse.c:1094
> #6  0x0000000000c74336 in x264_macroblock_analyse (h=0x7fffea255d60)
>    at encoder/analyse.c:3269
> #7  0x0000000000bf540d in x264_slice_write (h=0x7fffea255d60)
>    at encoder/encoder.c:2299
> #8  0x0000000000bf7d47 in x264_slices_write (h=0x7fffea255d60)
>    at encoder/encoder.c:2607
> #9  0x0000000000bff235 in x264_threadpool_thread (pool=0x7fffe95ee280)
>    at common/threadpool.c:69
> #10 0x00007ffff7587e9a in start_thread ()
>   from /lib/x86_64-linux-gnu/libpthread.so.0
> #11 0x00007ffff68a14bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
> #12 0x0000000000000000 in ?? ()
>
>
> My encoding settings (snippet from mediainfo -f output):
>
> Writing library/Name                     : x264
> Writing library/Version                  : core 125
> Encoding settings                        : cabac=1 / ref=3 /
> deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 /
> psy_rd=1.00:0.15 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 /
> 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-3 /
> threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1
> / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 /
> b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 /
> weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 /
> rc_lookahead=60 / rc=crf / mbtree=1 / crf=19.0 / qcomp=0.60 / qpmin=3 /
> qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
>
> I keep getting the segfaults at the same address. I was wondering if
> anyone knows how to investigate this further.

Replicate the problem with x264 itself and post a combination of
commandline options and source that reproduces the problem.

Jason