[x265] x265 crashes/gets stuck when giving more than '--slices 16'

[Michael Lackner]

Good morning!

I applied the patch and rebuilt the code (2.3+9-820f4327ddac in this case).

Invoking x265 on a 8292x3428 video with '--slices 20' now gives the following warning:

x265 [warning]: maxSlices can not be more than min(rows, MAX_NAL_UNITS-1), force set to 15

And it shows:

x265 [info]: Slices                              : 15

Just a laymans' question: I can see in source/encoder/nal.h, that MAX_NAL_UNITS is 16.
Should maxSlices really be MAX_NAL_UNITS-1 and not MAX_NAL_UNITS in this case? I'm just
asking because encoding with 16 slices seemed to work just fine before the patch.

Or are there issues with that?

Thanks!

Best,
Michael

-- 
Michael Lackner
Lehrstuhl für Informationstechnologie (CiT)
Montanuniversität Leoben
Tel.: +43 (0)3842/402-1505 | Mail: michael.lackner at unileoben.ac.at
Fax.: +43 (0)3842/402-1502 | Web: http://institute.unileoben.ac.at/infotech

On 03/14/2017 05:15 AM, Pradeep Ramachandran wrote:
> The limit check was checking against the wrong variable - it should've
> checked against slicesLimit and not against numRows.
> Can you please check if https://patches.videolan.org/patch/15905/ solves
> your issue?
> 
> Pradeep Ramachandran, PhD
> Solution Architect at www.multicorewareinc.com/
> Adjunct Faculty at www.cse.iitm.ac.in/
> pradeeprama.info/
> Ph:   +91 99627 82018
> 
> On Mon, Mar 6, 2017 at 4:30 PM, Michael Lackner <
> michael.lackner at unileoben.ac.at> wrote:
> 
>> Greetings,
>>
>> During my experimentation with creating a x265-based benchmark, I found
>> out that x265
>> segfaults and/or freezes when giving it more than 16 frame slices to
>> encode (e.g. --slices
>> 17 or --slices 20 or --slices 32 or similar).
>>
>> It seems easy to reproduce as well. In my case, I'm feeding raw 16-bit YUV
>> 4:4:4 content
>> to it (16-bit per channel, 48-bit per pixel, no chroma subsampling). The
>> resolution is
>> 8192×3428, an upscale of the free movie "Tears of Steel".
>>
>> It works perfectly fine with <=16 slices on all my systems though, here's
>> the ones I've
>> tested it on:
>>
>> Environments / software versions:
>>
>>   * x265 2.2+22-20217c8af8ac
>>
>>      CentOS 6.8 Linux x86_64, x265 built by GCC 6.2.0 + yasm 1.3.0
>>       => Segmentation fault
>>
>>   * x265 2.3+9-820f4327ddac
>>
>>     o CentOS 6.8 Linux x86_64, x265 built by GCC 6.2.0 + yasm 1.3.0
>>       => Segmentation fault
>>
>>     o FreeBSD 10.3 UNIX x86_64, x265 built by clang 3.8.1 + yasm 1.3.0
>>       => Process locks up in STOP state and becomes unkillable, even
>>          to SIGKILL
>>
>>     o Windows XP Professional x64 Edition (NT5.2), x265 built by
>>       MSVC2010 SP1, cl.exe 16.00.40219.01 + yasm 1.3.0
>>       => Terminates with %ERRORLEVEL% still set to 0, so this crash is
>>       completely uncaught
>>
>> As said, it works fine on all platforms with --slices [1..16].
>>
>> Here's some sample output of the crash from Linux, it's supposed to be a
>> 2-pass encode:
>>
>> 29163 Segmentation fault      (core dumped) nice -n19 x265 ./raw8k.yuv
>> --frames 500
>> --input-depth 16 --dither --input-res 8192x3428 --input-csp i444 -D 10
>> --fps 24 --slices
>> 20 -p veryslow --pmode --pme --open-gop --ref 6 --bframes 16 --b-pyramid
>> --weightb
>> --max-merge 5 --b-intra --bitrate 50000 --rect --amp --aq-mode 2 --no-sao
>> --qcomp 0.75
>> --no-strong-intra-smoothing --psy-rd 1.6 --psy-rdoq 5.0 --rdoq-level 1
>> --tu-inter-depth 4
>> --tu-intra-depth 4 --ctu 32 --max-tu-size 32 --pass 1 --slow-firstpass
>> --stats v.stats
>> --sar 1 --range full -o pass1.h265
>>
>> I don't have any debug builds of x265 right now and I don't really know
>> how to even do any
>> debugging, but if you can tell me if you need anything, I can always try
>> to build a debug
>> version to generate more helpful output / dump files.
>>
>> Or maybe 16 frame slices are supposed to be the maximum, but it's just not
>> handled
>> correctly yet?!
>>
>> Is there any information available on that behavior?
>>
>> --
>> Michael Lackner
>> Lehrstuhl für Informationstechnologie (CiT)
>> Montanuniversität Leoben
>> Tel.: +43 (0)3842/402-1505 | Mail: michael.lackner at unileoben.ac.at
>> Fax.: +43 (0)3842/402-1502 | Web: http://institute.unileoben.ac.
>> at/infotech
>> _______________________________________________
>> x265-devel mailing list
>> x265-devel at videolan.org
>> https://mailman.videolan.org/listinfo/x265-devel