[x265] Commit e3bd326a9f6f is a crasher

[Steve Borho]

On Tue, Aug 13, 2013 at 3:32 PM, Nikos Barkas <nikbar2004 at yahoo.com> wrote:

> I am getting crashes at the start of encoding after commit e3bd326a9f6f on
> Windows. I am building using GCC 4.7.1-TDM and the command I use to test is
> x265 --input stream.yuv  --width 720 --height 480 --rate 24 -o stream.h265
> --rect --max-merge 1 --hash 1 --wpp --tu-intra-depth 1 --tu-inter-depth 2
> --no-tskip -f 100
>
> stream.yuv is a 2GB file I use as standard test, of which I encode the 100
> first frames. The encoder crashes before any kind of progress information
> gets displayed, but after the header information has been shown. I tried
> building the RelWithDebInfo target and getting a stack backtrace and this
> is what I received:
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x00672f25 in x265_frame_init_lowres_core_ssse3 ()
>

Hmm, we must be feeding the assembly code bad inputs


> Trying the bt gdb command gives an endless loop of output like below,
> indicating a trashed stack (note, this is with debug info enabled):
> #0 0xbaadf00d in ?? ()
> #1 0xbaadf00d in ?? ()
> #2 0xbaadf00d in ?? ()
> ...
> #219 0xbaadf00d in ?? ()
> #220 0xbaadf00d in ?? ()
> #221 0xbaadf00d in ?? ()
> #222 0xbaadf00d in ?? ()
> and so on.
>
> My system is:
> Windows 7 Enterprise 32-bit
> Intel Corei7 2.8GHz
> 4GB RAM
> Intel HD Graphics
> 220 GB HDD
>
> I am not sure at this point what is required for this to get fixed; any
> ideas would be more than welcome. Should any additional data be needed,
> please let me know.
>
> Best regards
> Nikos (Vid01)
>

Thanks for the bug report.  In the short term the downscale planes are not
yet being used, so I can disable this again until it can be fully debugged.

-- 
Steve Borho
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.videolan.org/private/x265-devel/attachments/20130813/32b9ea97/attachment.html>