[x264-devel] segfault in x264_mb_encode_chroma_internal

[Miklos Juhasz]

Hi,

I've been encoding with Handbrake using the latest version of x264
(master, last commit 2012-05-18) on Ubuntu Linux 12.04 and experienced
quite frequently that the program stopped with general protection error.
Once it made X segfault and once even caused kernel panic. I am running
an i7 2600K, 12 threads.

Using debug symbols and gdb here's the backtrace:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff917fa700 (LWP 4401)]
0x0000000000c96d09 in x264_mb_encode_chroma_internal (chroma422=0, i_qp=21,
    b_inter=0, h=0x7fffea255d60) at encoder/macroblock.c:367

#0  0x0000000000c96d09 in x264_mb_encode_chroma_internal (chroma422=0,
    i_qp=21, b_inter=0, h=0x7fffea255d60) at encoder/macroblock.c:367
#1  x264_mb_encode_chroma (h=0x7fffea255d60, b_inter=0, i_qp=21)
    at encoder/macroblock.c:457
#2  0x0000000000c98636 in x264_macroblock_encode_internal (chroma=1,
    plane_count=1, h=0x7fffea255d60) at encoder/macroblock.c:888
#3  x264_macroblock_encode (h=0x7fffea255d60) at encoder/macroblock.c:927
#4  0x0000000000c62a1e in x264_rd_cost_mb (h=0x7fffea255d60, i_lambda2=3686)
    at encoder/rdo.c:162
#5  0x0000000000c696aa in x264_intra_rd (h=0x7fffea255d60,
a=0x7fff917f4180,
    i_satd_thresh=3162) at encoder/analyse.c:1094
#6  0x0000000000c74336 in x264_macroblock_analyse (h=0x7fffea255d60)
    at encoder/analyse.c:3269
#7  0x0000000000bf540d in x264_slice_write (h=0x7fffea255d60)
    at encoder/encoder.c:2299
#8  0x0000000000bf7d47 in x264_slices_write (h=0x7fffea255d60)
    at encoder/encoder.c:2607
#9  0x0000000000bff235 in x264_threadpool_thread (pool=0x7fffe95ee280)
    at common/threadpool.c:69
#10 0x00007ffff7587e9a in start_thread ()
   from /lib/x86_64-linux-gnu/libpthread.so.0
#11 0x00007ffff68a14bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
#12 0x0000000000000000 in ?? ()


My encoding settings (snippet from mediainfo -f output):

Writing library/Name                     : x264
Writing library/Version                  : core 125
Encoding settings                        : cabac=1 / ref=3 /
deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 /
psy_rd=1.00:0.15 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 /
8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-3 /
threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1
/ interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 /
b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 /
weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 /
rc_lookahead=60 / rc=crf / mbtree=1 / crf=19.0 / qcomp=0.60 / qpmin=3 /
qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00

I keep getting the segfaults at the same address. I was wondering if
anyone knows how to investigate this further.

Best regards,
Miklos