[x265] memory consumption when encoding 8K video

Thu Apr 6 08:24:32 CEST 2017

Already FullHD (1080p) can be tricky with a 32-bit build.

4K UHD works on a PC with 8 GB RAM, but 8K UHD may allocate around 12 GB,  
so I experienced heavy swapping.

Am 06.04.2017, 07:18 Uhr, schrieb Santhoshini Sekar  
<santhoshini at multicorewareinc.com>:

> Hi Yiran Li,
>
> We found such memory issues for 32-bit builds even with 4K content. It is
> very natural for a 32 bit build to fail for high resolution videos. We  
> have
> a bitbucket issue pointing to this (
> https://bitbucket.org/multicoreware/x265/issues/310/memory-error-crash-in-32-bit-compiler-for
> ).
>
> Thanks,
> Santhoshini
>
> On Thu, Apr 6, 2017 at 9:42 AM, YIRAN LI <mrfun.china at gmail.com> wrote:
>
>> Hi Pradeep,
>>
>> I'm now using x265.exe to reproduce this issue and yes I was able to
>> reproduce it.
>>
>> The thing is, x265.exe only accepts .yuv or y4m which store uncompressed
>> data so even only 1 second long clip takes more than 1 GB so I can't  
>> upload
>> them.
>>
>> If you want, you can use MSPaint to create a 8K jpg, then use ffmpeg
>> command to create a 1 second clip, here is the command line:
>> *                             ffmpeg.exe -framerate 30 -loop 1 -i  
>> 0001.jpg
>> -t 1 -pix_fmt yuv420p a.y4m*
>>
>> Once created, you can use x265.exe to convert it:
>>
>>
>> $ x265.exe --input a.y4m -o a.mp4
>>
>> y4m  [info]: 8192x4096 fps 30/1 i420p8 sar 1:1 frames 0 - 29 of 30
>>
>> raw  [info]: output file: a.mp4
>>
>> x265 [info]: HEVC encoder version 2.3
>>
>> x265 [info]: build info [Windows][GCC 5.4.0][32 bit] 8bit
>>
>> x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2
>> FMA3 LZCNT BMI2
>>
>> x265 [info]: Main profile, Level-6 (Main tier)
>>
>> x265 [info]: Thread pool created using 4 threads
>>
>> x265 [info]: Slices                              : 1
>>
>> x265 [info]: frame threads / pool features       : 2 / wpp(64 rows)
>>
>> x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
>>
>> x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1  
>> intra
>>
>> x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 2
>>
>> x265 [info]: Keyframe min / max / scenecut / bias: 25 / 250 / 40 / 5.00
>>
>> x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
>>
>> x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
>>
>> x265 [info]: References / ref-limit  cu / depth  : 3 / on / on
>>
>> x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.0 / 32 / 1
>>
>> x265 [info]: Rate Control / qCompress            : CRF-28.0 / 0.60
>>
>> x265 [info]: tools: rd=3 psy-rd=2.00 rskip signhide tmvp
>> strong-intra-smoothing
>>
>> x265 [info]: tools: lslices=8 deblock sao
>>
>> x265 [error]: malloc of size 37871616 failed
>>
>> x265 [error]: memory allocation failure, aborting encode
>>
>>
>> encoded 0 frames
>>
>> aborted at input frame 15, output frame 0
>>
>> I've tried almost all possible options:
>>
>> *x265.exe --input a.y4m -F 1 --lookahead-threads 1 --rc-lookahead 0
>> --bframes 0 --no-b-pyramid --ref 1 -o a.mp4*
>>
>> I
>> 'm not familiar with these parameters, could you tell me what possible
>> problems/disadvantages will the generated file have by
>> setting ref = 1, bframe = 0?
>>
>> Great thanks
>>
>> 2017-04-06 13:35 GMT+10:00 Pradeep Ramachandran <
>> pradeep at multicorewareinc.com>:
>>
>>> In general, I would recommend against a 32-bit build due to the fact  
>>> that
>>> several of our assembly optimizations (10-bit and 12-bit, in  
>>> particular)
>>> work only for 64-bit which would considerably restrict your encoding  
>>> speed.
>>> This is especially true for 8K where there is so many bits to deal  
>>> with!
>>>
>>> If you can share with us the command-line that you are trying, along  
>>> with
>>> a sample clip, we can try to reproduce the failure from our side.
>>>
>>> Pradeep Ramachandran, PhD
>>> Solution Architect at www.multicorewareinc.com/
>>> Adjunct Faculty at www.cse.iitm.ac.in/
>>> pradeeprama.info/
>>> Ph:   +91 99627 82018 <+91%2099627%2082018>
>>>
>>> On Thu, Apr 6, 2017 at 6:22 AM, YIRAN LI <mrfun.china at gmail.com> wrote:
>>>
>>>> Hi guys,
>>>>
>>>> I'm writing a program which can encode video using X265 API.  It's a  
>>>> 32
>>>> bit program.
>>>>
>>>> When the program runs to encode a 8K video, I can see every
>>>> time encoder_encode is called, the program eats about 100MB so that  
>>>> after
>>>> 12 frames are sent to encoder and when encoder_encode  is called for
>>>> another frame,  encoder_encode  returns failure. (at this time, about  
>>>> 1.5GB
>>>> in total is consumed by the encoder).
>>>>
>>>>
>>>> I guess it's caused by memory limit so compiled everything to 64bit,
>>>> this time everything went well and I could see maximum memory  
>>>> consumption
>>>> is about 6GB.
>>>>
>>>>
>>>> But I still need to fix the problem on 32bit because a lot of  
>>>> customers
>>>> use 32 bit version. Just want to know, is there any option, to make  
>>>> encoder
>>>> hold less frames or what ever option that can make encoder consume  
>>>> less
>>>> memory?
>>>>
>>>>
>>>> Thanks for suggestion.
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> x265-devel mailing list
>>>> x265-devel at videolan.org
>>>> https://mailman.videolan.org/listinfo/x265-devel
>>>>
>>>>
>>>
>>> _______________________________________________
>>> x265-devel mailing list
>>> x265-devel at videolan.org
>>> https://mailman.videolan.org/listinfo/x265-devel
>>>
>>>
>>
>> _______________________________________________
>> x265-devel mailing list
>> x265-devel at videolan.org
>> https://mailman.videolan.org/listinfo/x265-devel
>>
>>

-- 

Fun and success!
Mario *LigH* Rohkrämer
mailto:contact at ligh.de