[x264-devel] Segfault in x264_pixel_avg2_w16_sse2

[Ian Rickards]

Hi all,

There is a problem with the new SSE2 optimizations - specifically when bframes option is used.  Linux with C2D.

Starting program: /mnt/net/progs/x264/x264 --bframes=3 --pass=1 --bitrate=1200 topgear_720x576.yuv -o ttt.264
x264 [info]: file name gives 720x576
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 Cache64

Program received signal SIGSEGV, Segmentation fault.
0x080a27dc in x264_pixel_avg2_w16_sse2 ()
(gdb) where
#0  0x080a27dc in x264_pixel_avg2_w16_sse2 ()
#1  0x080a0774 in mc_luma_sse2 (dst=0xbffcc7c8 "ÿÿÿÿ", i_dst_stride=16,
    src=0xb7d172e3, i_src_stride=832, mvx=-1211546909, mvy=16,
    i_width=-1211010333, i_height=135023920) at common/x86/mc-c.c:139
#2  0x080a0774 in mc_luma_sse2 (dst=Cannot access memory at address 0x18
) at common/x86/mc-c.c:139
Previous frame inner to this frame (corrupt stack?)

Thanks

Ian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.videolan.org/pipermail/x264-devel/attachments/20080321/bc7c1fd2/attachment.htm