[x265] memory consumption when encoding 8K video

Thu Apr 6 07:18:40 CEST 2017

Hi Yiran Li,

We found such memory issues for 32-bit builds even with 4K content. It is
very natural for a 32 bit build to fail for high resolution videos. We have
a bitbucket issue pointing to this (
https://bitbucket.org/multicoreware/x265/issues/310/memory-error-crash-in-32-bit-compiler-for
).

Thanks,
Santhoshini

On Thu, Apr 6, 2017 at 9:42 AM, YIRAN LI <mrfun.china at gmail.com> wrote:

> Hi Pradeep,
>
> I'm now using x265.exe to reproduce this issue and yes I was able to
> reproduce it.
>
> The thing is, x265.exe only accepts .yuv or y4m which store uncompressed
> data so even only 1 second long clip takes more than 1 GB so I can't upload
> them.
>
> If you want, you can use MSPaint to create a 8K jpg, then use ffmpeg
> command to create a 1 second clip, here is the command line:
> *                             ffmpeg.exe -framerate 30 -loop 1 -i 0001.jpg
> -t 1 -pix_fmt yuv420p a.y4m*
>
> Once created, you can use x265.exe to convert it:
>
>
> $ x265.exe --input a.y4m -o a.mp4
>
> y4m  [info]: 8192x4096 fps 30/1 i420p8 sar 1:1 frames 0 - 29 of 30
>
> raw  [info]: output file: a.mp4
>
> x265 [info]: HEVC encoder version 2.3
>
> x265 [info]: build info [Windows][GCC 5.4.0][32 bit] 8bit
>
> x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2
> FMA3 LZCNT BMI2
>
> x265 [info]: Main profile, Level-6 (Main tier)
>
> x265 [info]: Thread pool created using 4 threads
>
> x265 [info]: Slices                              : 1
>
> x265 [info]: frame threads / pool features       : 2 / wpp(64 rows)
>
> x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
>
> x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
>
> x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 2
>
> x265 [info]: Keyframe min / max / scenecut / bias: 25 / 250 / 40 / 5.00
>
> x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
>
> x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
>
> x265 [info]: References / ref-limit  cu / depth  : 3 / on / on
>
> x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.0 / 32 / 1
>
> x265 [info]: Rate Control / qCompress            : CRF-28.0 / 0.60
>
> x265 [info]: tools: rd=3 psy-rd=2.00 rskip signhide tmvp
> strong-intra-smoothing
>
> x265 [info]: tools: lslices=8 deblock sao
>
> x265 [error]: malloc of size 37871616 failed
>
> x265 [error]: memory allocation failure, aborting encode
>
>
> encoded 0 frames
>
> aborted at input frame 15, output frame 0
>
> I've tried almost all possible options:
>
> *x265.exe --input a.y4m -F 1 --lookahead-threads 1 --rc-lookahead 0
> --bframes 0 --no-b-pyramid --ref 1 -o a.mp4*
>
> I
> 'm not familiar with these parameters, could you tell me what possible
> problems/disadvantages will the generated file have by
> setting ref = 1, bframe = 0?
>
> Great thanks
>
> 2017-04-06 13:35 GMT+10:00 Pradeep Ramachandran <
> pradeep at multicorewareinc.com>:
>
>> In general, I would recommend against a 32-bit build due to the fact that
>> several of our assembly optimizations (10-bit and 12-bit, in particular)
>> work only for 64-bit which would considerably restrict your encoding speed.
>> This is especially true for 8K where there is so many bits to deal with!
>>
>> If you can share with us the command-line that you are trying, along with
>> a sample clip, we can try to reproduce the failure from our side.
>>
>> Pradeep Ramachandran, PhD
>> Solution Architect at www.multicorewareinc.com/
>> Adjunct Faculty at www.cse.iitm.ac.in/
>> pradeeprama.info/
>> Ph:   +91 99627 82018 <+91%2099627%2082018>
>>
>> On Thu, Apr 6, 2017 at 6:22 AM, YIRAN LI <mrfun.china at gmail.com> wrote:
>>
>>> Hi guys,
>>>
>>> I'm writing a program which can encode video using X265 API.  It's a 32
>>> bit program.
>>>
>>> When the program runs to encode a 8K video, I can see every
>>> time encoder_encode is called, the program eats about 100MB so that after
>>> 12 frames are sent to encoder and when encoder_encode  is called for
>>> another frame,  encoder_encode  returns failure. (at this time, about 1.5GB
>>> in total is consumed by the encoder).
>>>
>>>
>>> I guess it's caused by memory limit so compiled everything to 64bit,
>>> this time everything went well and I could see maximum memory consumption
>>> is about 6GB.
>>>
>>>
>>> But I still need to fix the problem on 32bit because a lot of customers
>>> use 32 bit version. Just want to know, is there any option, to make encoder
>>> hold less frames or what ever option that can make encoder consume less
>>> memory?
>>>
>>>
>>> Thanks for suggestion.
>>>
>>>
>>>
>>> _______________________________________________
>>> x265-devel mailing list
>>> x265-devel at videolan.org
>>> https://mailman.videolan.org/listinfo/x265-devel
>>>
>>>
>>
>> _______________________________________________
>> x265-devel mailing list
>> x265-devel at videolan.org
>> https://mailman.videolan.org/listinfo/x265-devel
>>
>>
>
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20170406/93db96ab/attachment.html>