[x265] [PATCH] Add vbv-end tolerance check
Alex Giladi
alex.giladi at gmail.com
Tue Jun 12 15:10:39 CEST 2018
Hi Kalyan,
If this is the case, then everything is fine.
Best,
Alex.
On Tue, Jun 12, 2018 at 2:28 AM, Kalyan Goswami <kalyan at multicorewareinc.com
> wrote:
> Hi Alex,
>
> I think there is some misunderstanding of this email thread. I have
> checked the patch along with Ashok and Aruna, and my understanding is given
> below as a diagram. In this patch, we are introducing a small offset (5%)
> which aims the buffer fullness to be less the target by this offset. Since
> VBV is a probabilistic algorithm, this offset (depend upon the complexity
> of the video) provides little more confidence about the overshoot of the
> buffer.
>
> In addition to that, this offset is always negative (in terms of buffer
> fullness). Hence, it should not break your existing system.
> Please let us know if we are missing something.
>
>
> Thanks,
>
> Kalyan Goswami, PhD
> Video Architect @ MulticoreWare
> http: <http://www.multicorewareinc.com/>//www.multicorewareinc.com
> <http://www.multicorewareinc.com/>
> +91 9884989331
>
> On Mon, Jun 11, 2018 at 10:56 PM, Alex Giladi <alex.giladi at gmail.com>
> wrote:
>
>> This may still break HRD compliance -- imagine you have a large amount of
>> chunks (hundreds or thousands per movie) which you encode in parallel.
>> Under VBV-based (VBV or VBV+CRF) rate control you try running as close to
>> the buffer capacity as you can. If uncoordinated overshoot of e.g. 4% in
>> buffer fulness occurs sufficiently frequently, chunks towards the end may
>> overflow as we developed a drift between the real CPB fulness and the CPB
>> fulness assumed by x265 encoding a chunk.
>>
>> Hence this patch breaks HRD compatibility for parallel encoding of any
>> sufficiently complex long-form content. To make things more interesting,
>> this breakage is not deterministic.
>>
>> You can allow fullness which is less than vbv-end. This would mean that
>> towards the end of the movie you may be under-utilizing the CPB, but you
>> will not generate incompliant streams.
>>
>> On Mon, Jun 11, 2018 at 1:24 AM, Aruna Matheswaran <
>> aruna at multicorewareinc.com> wrote:
>>
>>>
>>>
>>> On Mon, Jun 11, 2018 at 3:34 AM, Alex Giladi <alex.giladi at gmail.com>
>>> wrote:
>>>
>>>> Does this "5% tolerance" imply that CPB fullness as defined by vbv-end
>>>> can be exceeded by 5%?
>>>>
>>>
>>> yes, the CPB fullness can vary by +/- 5 % from the value defined by
>>> vbv-end.
>>>
>>>>
>>>> On Fri, Jun 8, 2018 at 6:39 AM, <aruna at multicorewareinc.com> wrote:
>>>>
>>>>> # HG changeset patch
>>>>> # User Aruna Matheswaran <aruna at multicorewareinc.com>
>>>>> # Date 1523005500 -19800
>>>>> # Fri Apr 06 14:35:00 2018 +0530
>>>>> # Node ID ed853c4af6710a991d0cdf4bf68e00fe32edaacb
>>>>> # Parent 182914e1d201395d152e310db7f5cf29ab3c787e
>>>>> Add vbv-end tolerance check
>>>>>
>>>>> This will attempt to keep the desired fraction of the target buffer (
>>>>> specified
>>>>> by vbv-end ) empty within a defined error margin of 5%.
>>>>>
>>>>> diff -r 182914e1d201 -r ed853c4af671 source/encoder/ratecontrol.cpp
>>>>> --- a/source/encoder/ratecontrol.cpp Wed Mar 14 12:30:28 2018 +0530
>>>>> +++ b/source/encoder/ratecontrol.cpp Fri Apr 06 14:35:00 2018 +0530
>>>>> @@ -1282,6 +1282,12 @@
>>>>> m_predictedBits = m_totalBits;
>>>>> updateVbvPlan(enc);
>>>>> rce->bufferFill = m_bufferFill;
>>>>> + rce->vbvEndAdj = false;
>>>>> + if (m_param->vbvBufferEnd && rce->encodeOrder >=
>>>>> m_param->vbvEndFrameAdjust * m_param->totalFrames)
>>>>> + {
>>>>> + rce->vbvEndAdj = true;
>>>>> + rce->targetFill = 0;
>>>>> + }
>>>>>
>>>>> int mincr = enc->m_vps.ptl.minCrForLevel;
>>>>> /* Profiles above Main10 don't require maxAU size check, so
>>>>> just set the maximum to a large value. */
>>>>> @@ -2173,12 +2179,12 @@
>>>>> curBits = predictSize(&m_pred[predType],
>>>>> frameQ[type], (double)satd);
>>>>> bufferFillCur -= curBits;
>>>>> }
>>>>> - if (m_param->vbvBufferEnd && rce->encodeOrder >=
>>>>> m_param->vbvEndFrameAdjust * m_param->totalFrames)
>>>>> + if (rce->vbvEndAdj)
>>>>> {
>>>>> bool loopBreak = false;
>>>>> double bufferDiff = m_param->vbvBufferEnd -
>>>>> (m_bufferFill / m_bufferSize);
>>>>> - targetFill = m_bufferFill + m_bufferSize *
>>>>> (bufferDiff / (m_param->totalFrames - rce->encodeOrder));
>>>>> - if (bufferFillCur < targetFill)
>>>>> + rce->targetFill = m_bufferFill + m_bufferSize *
>>>>> (bufferDiff / (m_param->totalFrames - rce->encodeOrder));
>>>>> + if (bufferFillCur < rce->targetFill)
>>>>> {
>>>>> q *= 1.01;
>>>>> loopTerminate |= 1;
>>>>> @@ -2421,6 +2427,7 @@
>>>>> double rcTol = bufferLeftPlanned / m_param->frameNumThreads *
>>>>> m_rateTolerance;
>>>>> int32_t encodedBitsSoFar = 0;
>>>>> double accFrameBits = predictRowsSizeSum(curFrame, rce,
>>>>> qpVbv, encodedBitsSoFar);
>>>>> + double vbvEndBias = 0.95;
>>>>>
>>>>> /* * Don't increase the row QPs until a sufficent amount of
>>>>> the bits of
>>>>> * the frame have been processed, in case a flat area at the
>>>>> top of the
>>>>> @@ -2442,7 +2449,8 @@
>>>>> while (qpVbv < qpMax
>>>>> && (((accFrameBits > rce->frameSizePlanned + rcTol) ||
>>>>> (rce->bufferFill - accFrameBits <
>>>>> bufferLeftPlanned * 0.5) ||
>>>>> - (accFrameBits > rce->frameSizePlanned && qpVbv <
>>>>> rce->qpNoVbv))
>>>>> + (accFrameBits > rce->frameSizePlanned && qpVbv <
>>>>> rce->qpNoVbv) ||
>>>>> + (rce->vbvEndAdj && ((rce->bufferFill -
>>>>> accFrameBits) < (rce->targetFill * vbvEndBias))))
>>>>> && (!m_param->rc.bStrictCbr ? 1 : abrOvershoot >
>>>>> 0.1)))
>>>>> {
>>>>> qpVbv += stepSize;
>>>>> @@ -2453,7 +2461,8 @@
>>>>> while (qpVbv > qpMin
>>>>> && (qpVbv > curEncData.m_rowStat[0].rowQp ||
>>>>> m_singleFrameVbv)
>>>>> && (((accFrameBits < rce->frameSizePlanned * 0.8f &&
>>>>> qpVbv <= prevRowQp)
>>>>> - || accFrameBits < (rce->bufferFill - m_bufferSize
>>>>> + m_bufferRate) * 1.1)
>>>>> + || accFrameBits < (rce->bufferFill - m_bufferSize
>>>>> + m_bufferRate) * 1.1
>>>>> + || (rce->vbvEndAdj && ((rce->bufferFill -
>>>>> accFrameBits) > (rce->targetFill * vbvEndBias))))
>>>>> && (!m_param->rc.bStrictCbr ? 1 : abrOvershoot <
>>>>> 0)))
>>>>> {
>>>>> qpVbv -= stepSize;
>>>>> diff -r 182914e1d201 -r ed853c4af671 source/encoder/ratecontrol.h
>>>>> --- a/source/encoder/ratecontrol.h Wed Mar 14 12:30:28 2018 +0530
>>>>> +++ b/source/encoder/ratecontrol.h Fri Apr 06 14:35:00 2018 +0530
>>>>> @@ -82,6 +82,8 @@
>>>>> double rowCplxrSum;
>>>>> double qpNoVbv;
>>>>> double bufferFill;
>>>>> + double targetFill;
>>>>> + bool vbvEndAdj;
>>>>> double frameDuration;
>>>>> double clippedDuration;
>>>>> double frameSizeEstimated; /* hold frameSize, updated from cu
>>>>> level vbv rc */
>>>>>
>>>>> _______________________________________________
>>>>> x265-devel mailing list
>>>>> x265-devel at videolan.org
>>>>> https://mailman.videolan.org/listinfo/x265-devel
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> x265-devel mailing list
>>>> x265-devel at videolan.org
>>>> https://mailman.videolan.org/listinfo/x265-devel
>>>>
>>>>
>>>
>>> _______________________________________________
>>> x265-devel mailing list
>>> x265-devel at videolan.org
>>> https://mailman.videolan.org/listinfo/x265-devel
>>>
>>>
>>
>> _______________________________________________
>> x265-devel mailing list
>> x265-devel at videolan.org
>> https://mailman.videolan.org/listinfo/x265-devel
>>
>>
>
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20180612/9af05e21/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 5990 bytes
Desc: not available
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20180612/9af05e21/attachment-0001.png>
More information about the x265-devel
mailing list