<div dir="ltr">This may still break HRD compliance -- imagine you have a large amount of chunks (hundreds or thousands per movie) which you encode in parallel. <div>Under VBV-based (VBV or VBV+CRF) rate control you try running as close to the buffer capacity as you can. If uncoordinated overshoot of e.g. 4% in buffer fulness occurs sufficiently frequently, chunks towards the end may overflow as we developed a drift between the real CPB fulness and the CPB fulness assumed by x265 encoding a chunk.</div><div><br></div><div>Hence this patch breaks HRD compatibility for parallel encoding of any sufficiently complex long-form content. To make things more interesting, this breakage is not deterministic.</div><div><br></div><div>You can allow fullness which is less than vbv-end. This would mean that towards the end of the movie you may be under-utilizing the CPB, but you will not generate incompliant streams.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jun 11, 2018 at 1:24 AM, Aruna Matheswaran <span dir="ltr"><<a href="mailto:aruna@multicorewareinc.com" target="_blank">aruna@multicorewareinc.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Mon, Jun 11, 2018 at 3:34 AM, Alex Giladi <span dir="ltr"><<a href="mailto:alex.giladi@gmail.com" target="_blank">alex.giladi@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Does this "5% tolerance" imply that CPB fullness as defined by vbv-end can be exceeded by 5%?</div></blockquote><div><br></div></span><div>yes, the CPB fullness can vary by +/- 5 % from the value defined by vbv-end. </div><div><div class="h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="m_8060553625789790075gmail-h5">On Fri, Jun 8, 2018 at 6:39 AM, <span dir="ltr"><<a href="mailto:aruna@multicorewareinc.com" target="_blank">aruna@multicorewareinc.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="m_8060553625789790075gmail-h5"># HG changeset patch<br>
# User Aruna Matheswaran <<a href="mailto:aruna@multicorewareinc.com" target="_blank">aruna@multicorewareinc.com</a>><br>
# Date 1523005500 -19800<br>
# Fri Apr 06 14:35:00 2018 +0530<br>
# Node ID ed853c4af6710a991d0cdf4bf68e00<wbr>fe32edaacb<br>
# Parent 182914e1d201395d152e310db7f5cf<wbr>29ab3c787e<br>
Add vbv-end tolerance check<br>
<br>
This will attempt to keep the desired fraction of the target buffer ( specified<br>
by vbv-end ) empty within a defined error margin of 5%.<br>
<br>
diff -r 182914e1d201 -r ed853c4af671 source/encoder/ratecontrol.cpp<br>
--- a/source/encoder/ratecontrol.c<wbr>pp Wed Mar 14 12:30:28 2018 +0530<br>
+++ b/source/encoder/ratecontrol.c<wbr>pp Fri Apr 06 14:35:00 2018 +0530<br>
@@ -1282,6 +1282,12 @@<br>
m_predictedBits = m_totalBits;<br>
updateVbvPlan(enc);<br>
rce->bufferFill = m_bufferFill;<br>
+ rce->vbvEndAdj = false;<br>
+ if (m_param->vbvBufferEnd && rce->encodeOrder >= m_param->vbvEndFrameAdjust * m_param->totalFrames)<br>
+ {<br>
+ rce->vbvEndAdj = true;<br>
+ rce->targetFill = 0;<br>
+ }<br>
<br>
int mincr = enc->m_vps.ptl.minCrForLevel;<br>
/* Profiles above Main10 don't require maxAU size check, so just set the maximum to a large value. */<br>
@@ -2173,12 +2179,12 @@<br>
curBits = predictSize(&m_pred[predType], frameQ[type], (double)satd);<br>
bufferFillCur -= curBits;<br>
}<br>
- if (m_param->vbvBufferEnd && rce->encodeOrder >= m_param->vbvEndFrameAdjust * m_param->totalFrames)<br>
+ if (rce->vbvEndAdj)<br>
{<br>
bool loopBreak = false;<br>
double bufferDiff = m_param->vbvBufferEnd - (m_bufferFill / m_bufferSize);<br>
- targetFill = m_bufferFill + m_bufferSize * (bufferDiff / (m_param->totalFrames - rce->encodeOrder));<br>
- if (bufferFillCur < targetFill)<br>
+ rce->targetFill = m_bufferFill + m_bufferSize * (bufferDiff / (m_param->totalFrames - rce->encodeOrder));<br>
+ if (bufferFillCur < rce->targetFill)<br>
{<br>
q *= 1.01;<br>
loopTerminate |= 1;<br>
@@ -2421,6 +2427,7 @@<br>
double rcTol = bufferLeftPlanned / m_param->frameNumThreads * m_rateTolerance;<br>
int32_t encodedBitsSoFar = 0;<br>
double accFrameBits = predictRowsSizeSum(curFrame, rce, qpVbv, encodedBitsSoFar);<br>
+ double vbvEndBias = 0.95;<br>
<br>
/* * Don't increase the row QPs until a sufficent amount of the bits of<br>
* the frame have been processed, in case a flat area at the top of the<br>
@@ -2442,7 +2449,8 @@<br>
while (qpVbv < qpMax<br>
&& (((accFrameBits > rce->frameSizePlanned + rcTol) ||<br>
(rce->bufferFill - accFrameBits < bufferLeftPlanned * 0.5) ||<br>
- (accFrameBits > rce->frameSizePlanned && qpVbv < rce->qpNoVbv))<br>
+ (accFrameBits > rce->frameSizePlanned && qpVbv < rce->qpNoVbv) ||<br>
+ (rce->vbvEndAdj && ((rce->bufferFill - accFrameBits) < (rce->targetFill * vbvEndBias))))<br>
&& (!m_param->rc.bStrictCbr ? 1 : abrOvershoot > 0.1)))<br>
{<br>
qpVbv += stepSize;<br>
@@ -2453,7 +2461,8 @@<br>
while (qpVbv > qpMin<br>
&& (qpVbv > curEncData.m_rowStat[0].rowQp || m_singleFrameVbv)<br>
&& (((accFrameBits < rce->frameSizePlanned * 0.8f && qpVbv <= prevRowQp)<br>
- || accFrameBits < (rce->bufferFill - m_bufferSize + m_bufferRate) * 1.1)<br>
+ || accFrameBits < (rce->bufferFill - m_bufferSize + m_bufferRate) * 1.1<br>
+ || (rce->vbvEndAdj && ((rce->bufferFill - accFrameBits) > (rce->targetFill * vbvEndBias))))<br>
&& (!m_param->rc.bStrictCbr ? 1 : abrOvershoot < 0)))<br>
{<br>
qpVbv -= stepSize;<br>
diff -r 182914e1d201 -r ed853c4af671 source/encoder/ratecontrol.h<br>
--- a/source/encoder/ratecontrol.h<wbr> Wed Mar 14 12:30:28 2018 +0530<br>
+++ b/source/encoder/ratecontrol.h<wbr> Fri Apr 06 14:35:00 2018 +0530<br>
@@ -82,6 +82,8 @@<br>
double rowCplxrSum;<br>
double qpNoVbv;<br>
double bufferFill;<br>
+ double targetFill;<br>
+ bool vbvEndAdj;<br>
double frameDuration;<br>
double clippedDuration;<br>
double frameSizeEstimated; /* hold frameSize, updated from cu level vbv rc */<br>
<br></div></div>______________________________<wbr>_________________<br>
x265-devel mailing list<br>
<a href="mailto:x265-devel@videolan.org" target="_blank">x265-devel@videolan.org</a><br>
<a href="https://mailman.videolan.org/listinfo/x265-devel" rel="noreferrer" target="_blank">https://mailman.videolan.org/l<wbr>istinfo/x265-devel</a><br>
<br></blockquote></div><br></div>
<br>______________________________<wbr>_________________<br>
x265-devel mailing list<br>
<a href="mailto:x265-devel@videolan.org" target="_blank">x265-devel@videolan.org</a><br>
<a href="https://mailman.videolan.org/listinfo/x265-devel" rel="noreferrer" target="_blank">https://mailman.videolan.org/l<wbr>istinfo/x265-devel</a><br>
<br></blockquote></div></div></div><br></div></div>
<br>______________________________<wbr>_________________<br>
x265-devel mailing list<br>
<a href="mailto:x265-devel@videolan.org">x265-devel@videolan.org</a><br>
<a href="https://mailman.videolan.org/listinfo/x265-devel" rel="noreferrer" target="_blank">https://mailman.videolan.org/<wbr>listinfo/x265-devel</a><br>
<br></blockquote></div><br></div>