[x265] Crash with 407d8c61698d

[Steve Borho]

On Mon, Aug 26, 2013 at 2:19 PM, Steve Borho <steve at borho.org> wrote:

>
>
>
> On Mon, Aug 26, 2013 at 1:37 PM, Steve Borho <steve at borho.org> wrote:
>
>>
>>
>>
>> On Mon, Aug 26, 2013 at 1:13 PM, Nikos Barkas <nikbar2004 at yahoo.com>wrote:
>>
>>> Commit 407d8c61698d crashes for me after displaying the header
>>> information and before starting to actually encode data. Below is the
>>> backtrace from gdb from the RelWithDebInfo target. Compiling on Win7
>>> 32-bit, GCC 4.7.1-TDM.
>>>
>>>
>>>
>>> Program received signal SIGSEGV, Segmentation fault.
>>> 0x00477d6a in filterRowH (
>>>     src=0x5341790 "ÖÖÖxxxxxOOxOOOOOOOOUOOOOOOO", 'U' <repeats 17 times>,
>>> "UUUÜYY
>>> Y____YYÜYÜÜUUUUU", 'U' <repeats 28 times>, "UU", 'O' <repeats 104
>>> times>...,
>>>     srcStride=880, midA=0x9f2e900, midB=0x9fde900, midC=0xa08e900,
>>>     midStride=720, dstA=0xa2c1790 "", dstB=0xa501790 "", dstC=0xa741790
>>> "",
>>>     width=720, height=64, marginX=80, marginY=80, row=7, isLastRow=1)
>>>     at c:/DriveD/x265Project/x265/source/common/ipfilter.cpp:591
>>> 591                 ::memcpy(src - marginX + (height + y) * srcStride,
>>> src - mar
>>> ginX + (height - 1) * srcStride, sizeof(pixel) * (width + (marginX <<
>>> 1)));
>>> (gdb) bt
>>> #0  0x00477d6a in filterRowH (
>>>     src=0x5341790 "ÖÖÖxxxxxOOxOOOOOOOOUOOOOOOO", 'U' <repeats 17 times>,
>>> "UUUÜYY
>>> Y____YYÜYÜÜUUUUU", 'U' <repeats 28 times>, "UU", 'O' <repeats 104
>>> times>...,
>>>     srcStride=880, midA=0x9f2e900, midB=0x9fde900, midC=0xa08e900,
>>>     midStride=720, dstA=0xa2c1790 "", dstB=0xa501790 "", dstC=0xa741790
>>> "",
>>>     width=720, height=64, marginX=80, marginY=80, row=7, isLastRow=1)
>>>     at c:/DriveD/x265Project/x265/source/common/ipfilter.cpp:591
>>> #1  0x0047a463 in x265::MotionReference::generateReferencePlanes (
>>>     this=0x8eb2908)
>>>     at c:/DriveD/x265Project/x265/source/common/reference.cpp:197
>>> #2  0x00445e54 in TComPicYuv::generateMotionReference (this=0x4853f10,
>>>     pool=0x21bf970, w=0x0)
>>>     at
>>> c:/DriveD/x265Project/x265/source/Lib/TLibCommon/TComPicYuv.cpp:264
>>> #3  0x0040d812 in x265::FrameEncoder::compressFrame (this=0x21bfe80, pic=
>>>     0x4980078)
>>>     at c:/DriveD/x265Project/x265/source/encoder/frameencoder.cpp:436
>>> #4  0x0022fb08 in ?? ()
>>> Backtrace stopped: previous frame inner to this frame (corrupt stack?)
>>> (gdb)
>>>
>>>
>> Yes, I'm getting the same thing here.  Still investigating.  Worst case
>> I'll need to backout  63e0736c1f87
>>
>
> I've pushed a fix for this.
>

This no longer crashes, but MD5 hashes of recon pictures are no longer
matching decoder pictures following 63e0736c1f87, so more fixes remain to
be done on it.

-- 
Steve Borho
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.videolan.org/private/x265-devel/attachments/20130826/d41c6ba9/attachment.html>