[x264-devel] AddressSanitizer: Invalid Read at common/x86/mc-c.c:772

[Hongxu Chen]

Hi,

    x264 (sandbox version, HEAD d4099dd4c722f52c4f3c14575d7d39eb8fadb97f)
suffers from an invalid read error which may result in a segfault.
    An ASAN instrumented x264 reports error messages like the following:

$ ./x264-asan/install/bin/x264 --threads 1 read_mc-c.c:772_1.avi -o
/dev/null

[avi @ 0x61b000000080] crazy start time, iam scared, giving up
[avi @ 0x61b000000080] Something went wrong during header parsing, tag 0000
has size 808464432, I will ignore it and try to continue anyway.
lavf [info]: 667696x16p 0:1 @ 1/1 fps (vfr)
resize [warning]: converting from pal8 to rgb24
resize [warning]: converting from rgb24 to yuv420p
x264 [warning]: frame MB size (41731x1) > level limit (139264)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3
BMI2 AVX2
x264 [info]: profile Progressive High, level 6.2, 4:2:0, 8-bit
ASAN:DEADLYSIGNAL
=================================================================
==17224==ERROR: AddressSanitizer: SEGV on unknown address 0x7f71d989f2c0
(pc 0x0000007f1a77 bp 0x000000008008 sp 0x7ffdaef27240 T0)
==17224==The signal is caused by a READ memory access.
    #0 0x7f1a76 in mbtree_propagate_list_avx2
/home/hongxu/work/x264/x264-asan/common/x86/mc-c.c:772:1
    #1 0x74b89e in macroblock_tree_propagate
/home/hongxu/work/x264/x264-asan/encoder/slicetype.c:1071:9
    #2 0x723183 in macroblock_tree
/home/hongxu/work/x264/x264-asan/encoder/slicetype.c:1163:9
    #3 0x720338 in x264_8_slicetype_analyse
/home/hongxu/work/x264/x264-asan/encoder/slicetype.c:1671:9
    #4 0x5a84c9 in x264_8_lookahead_get_frames
/home/hongxu/work/x264/x264-asan/encoder/lookahead.c:245:13
    #5 0x57cd9e in x264_8_encoder_encode
/home/hongxu/work/x264/x264-asan/encoder/encoder.c:3361:9
    #6 0x5fefeb in x264_stack_align
(/home/hongxu/work/x264/x264-asan/install/bin/x264+0x5fefeb)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV
/home/hongxu/work/x264/x264-asan/common/x86/mc-c.c:772:1 in
mbtree_propagate_list_avx2
==17224==ABORTING
[1]    17224 abort      ./x264-asan/install/bin/x264 --threads 1
read_mc-c.c:772_1.avi -o /dev/null

$ ./x264-asan/install/bin/x264 --version
x264 0.157.2969 d4099dd
(libswscale 4.8.100)
(libavformat 57.83.100)
built on Apr  4 2019, gcc: 4.2.1 Compatible Clang 4.0.1
(tags/RELEASE_401/final)
x264 configuration: --chroma-format=all
libx264 configuration: --chroma-format=all
x264 license: GPL version 2 or later
libswscale/libavformat license: GPL version 2 or later

Best Regards,
Hongxu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x264-devel/attachments/20190404/b285013c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: read_mc-c.c:772_2.avi
Type: video/x-msvideo
Size: 300 bytes
Desc: not available
URL: <http://mailman.videolan.org/pipermail/x264-devel/attachments/20190404/b285013c/attachment.avi>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: read_mc-c.c:772_3.avi
Type: video/x-msvideo
Size: 1195 bytes
Desc: not available
URL: <http://mailman.videolan.org/pipermail/x264-devel/attachments/20190404/b285013c/attachment-0001.avi>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: read_mc-c.c:772_1.avi
Type: video/x-msvideo
Size: 356 bytes
Desc: not available
URL: <http://mailman.videolan.org/pipermail/x264-devel/attachments/20190404/b285013c/attachment-0002.avi>