<div dir="ltr">Hi Alex,<br><br>a. The software that uses the x265 library would trigger the re-encode<br>b. For the CRF+VBV 2pass run, it will drop performance on the re-encoding. It will not have an impact on the others<br>c. By default, it is not triggered. And we are not planning to add a command-line option as it just re-encodes some selected gops instead of all the frames. And it will work in a CRF or CRF+VBV mode. Without crf, it will not work<br><br>Thanks,<br>Mahesh<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Aug 10, 2021 at 1:55 AM Alex Giladi <<a href="mailto:alex.giladi@gmail.com">alex.giladi@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Thank you, Mahesh!<br>
Can you please explain (a) what triggers the re-encode, (b) what is<br>
the impact on run time, (c) is it triggered by default or are there<br>
plans to add a command-line option and, lastly, why doesn't it work in<br>
a VBV or CRF+VBV mode?<br>
Best,<br>
Alex.<br>
<br>
On Sun, Aug 8, 2021 at 10:37 PM Mahesh Pittala<br>
<<a href="mailto:mahesh@multicorewareinc.com" target="_blank">mahesh@multicorewareinc.com</a>> wrote:<br>
><br>
> Hi Alex,<br>
><br>
> This patch is to re-encode some separated gops to improve the compression efficiency.<br>
><br>
> Thanks,<br>
> Mahesh<br>
><br>
> On Fri, Aug 6, 2021 at 6:30 AM Alex Giladi <<a href="mailto:alex.giladi@gmail.com" target="_blank">alex.giladi@gmail.com</a>> wrote:<br>
>><br>
>> Hi Mahesh,<br>
>> What is the actual modification of crf in the patch?<br>
>> Best,<br>
>> Alex<br>
>><br>
>><br>
>> On Thu, Aug 5, 2021, 2:15 AM Mahesh Pittala <<a href="mailto:mahesh@multicorewareinc.com" target="_blank">mahesh@multicorewareinc.com</a>> wrote:<br>
>>><br>
>>> From 5065d31b4bfe36e05c94dd79db119cd1fd23aa23 Mon Sep 17 00:00:00 2001<br>
>>> From: lwWang <<a href="mailto:liwei@multicorewareinc.com" target="_blank">liwei@multicorewareinc.com</a>><br>
>>> Date: Fri, 25 Jun 2021 17:17:03 +0800<br>
>>> Subject: [PATCH] Modify the crf 2 pass<br>
>>><br>
>>> Signed-off-by: lwWang <<a href="mailto:liwei@multicorewareinc.com" target="_blank">liwei@multicorewareinc.com</a>><br>
>>> ---<br>
>>> source/common/frame.h | 1 +<br>
>>> source/encoder/encoder.cpp | 1 +<br>
>>> source/encoder/ratecontrol.cpp | 235 ++++++++++++++++-----------------<br>
>>> source/encoder/ratecontrol.h | 3 +<br>
>>> 4 files changed, 119 insertions(+), 121 deletions(-)<br>
>>><br>
>>> diff --git a/source/common/frame.h b/source/common/frame.h<br>
>>> index dc5bbacf7..ac1185e81 100644<br>
>>> --- a/source/common/frame.h<br>
>>> +++ b/source/common/frame.h<br>
>>> @@ -70,6 +70,7 @@ struct RcStats<br>
>>> double count[4];<br>
>>> double offset[4];<br>
>>> double bufferFillFinal;<br>
>>> + int64_t currentSatd;<br>
>>> };<br>
>>><br>
>>> class Frame<br>
>>> diff --git a/source/encoder/encoder.cpp b/source/encoder/encoder.cpp<br>
>>> index 003011554..227933e8a 100644<br>
>>> --- a/source/encoder/encoder.cpp<br>
>>> +++ b/source/encoder/encoder.cpp<br>
>>> @@ -2216,6 +2216,7 @@ int Encoder::encode(const x265_picture* pic_in, x265_picture* pic_out)<br>
>>> outFrame->m_rcData->iCuCount = outFrame->m_encData->m_frameStats.percent8x8Intra * m_rateControl->m_ncu;<br>
>>> outFrame->m_rcData->pCuCount = outFrame->m_encData->m_frameStats.percent8x8Inter * m_rateControl->m_ncu;<br>
>>> outFrame->m_rcData->skipCuCount = outFrame->m_encData->m_frameStats.percent8x8Skip * m_rateControl->m_ncu;<br>
>>> + outFrame->m_rcData->currentSatd = curEncoder->m_rce.coeffBits;<br>
>>> }<br>
>>><br>
>>> /* Allow this frame to be recycled if no frame encoders are using it for reference */<br>
>>> diff --git a/source/encoder/ratecontrol.cpp b/source/encoder/ratecontrol.cpp<br>
>>> index e9ceccc24..28345a1e9 100644<br>
>>> --- a/source/encoder/ratecontrol.cpp<br>
>>> +++ b/source/encoder/ratecontrol.cpp<br>
>>> @@ -1116,123 +1116,106 @@ bool RateControl::initPass2()<br>
>>> uint64_t allAvailableBits = uint64_t(m_param->rc.bitrate * 1000. * m_numEntries * m_frameDuration);<br>
>>> int startIndex, framesCount, endIndex;<br>
>>> int fps = X265_MIN(m_param->keyframeMax, (int)(m_fps + 0.5));<br>
>>> + int distance = fps << 1;<br>
>>> + distance = distance > m_param->keyframeMax ? (m_param->keyframeMax << 1) : m_param->keyframeMax;<br>
>>> startIndex = endIndex = framesCount = 0;<br>
>>> - int diffQp = 0;<br>
>>> double targetBits = 0;<br>
>>> double expectedBits = 0;<br>
>>> - for (startIndex = m_start, endIndex = m_start; endIndex < m_numEntries; endIndex++)<br>
>>> + double targetBits2 = 0;<br>
>>> + double expectedBits2 = 0;<br>
>>> + double cpxSum = 0;<br>
>>> + double cpxSum2 = 0;<br>
>>> +<br>
>>> + if (m_param->rc.rateControlMode == X265_RC_ABR)<br>
>>> {<br>
>>> - allConstBits += m_rce2Pass[endIndex].miscBits;<br>
>>> - allCodedBits += m_rce2Pass[endIndex].coeffBits + m_rce2Pass[endIndex].mvBits;<br>
>>> - if (m_param->rc.rateControlMode == X265_RC_CRF)<br>
>>> + for (startIndex = m_start, endIndex = m_start; endIndex < m_numEntries; endIndex++)<br>
>>> {<br>
>>> - framesCount = endIndex - startIndex + 1;<br>
>>> - diffQp += int (m_rce2Pass[endIndex].qpaRc - m_rce2Pass[endIndex].qpNoVbv);<br>
>>> - if (framesCount > fps)<br>
>>> - diffQp -= int (m_rce2Pass[endIndex - fps].qpaRc - m_rce2Pass[endIndex - fps].qpNoVbv);<br>
>>> - if (framesCount >= fps)<br>
>>> - {<br>
>>> - if (diffQp >= 1)<br>
>>> - {<br>
>>> - if (!m_isQpModified && endIndex > fps)<br>
>>> - {<br>
>>> - double factor = 2;<br>
>>> - double step = 0;<br>
>>> - if (endIndex + fps >= m_numEntries)<br>
>>> - {<br>
>>> - m_start = endIndex - (endIndex % fps);<br>
>>> - return true;<br>
>>> - }<br>
>>> - for (int start = endIndex + 1; start <= endIndex + fps && start < m_numEntries; start++)<br>
>>> - {<br>
>>> - RateControlEntry *rce = &m_rce2Pass[start];<br>
>>> - targetBits += qScale2bits(rce, x265_qp2qScale(rce->qpNoVbv));<br>
>>> - expectedBits += qScale2bits(rce, rce->qScale);<br>
>>> - }<br>
>>> - if (expectedBits < 0.95 * targetBits)<br>
>>> - {<br>
>>> - m_isQpModified = true;<br>
>>> - m_isGopReEncoded = true;<br>
>>> - while (endIndex + fps < m_numEntries)<br>
>>> - {<br>
>>> - step = pow(2, factor / 6.0);<br>
>>> - expectedBits = 0;<br>
>>> - for (int start = endIndex + 1; start <= endIndex + fps; start++)<br>
>>> - {<br>
>>> - RateControlEntry *rce = &m_rce2Pass[start];<br>
>>> - rce->newQScale = rce->qScale / step;<br>
>>> - X265_CHECK(rce->newQScale >= 0, "new Qscale is negative\n");<br>
>>> - expectedBits += qScale2bits(rce, rce->newQScale);<br>
>>> - rce->newQp = x265_qScale2qp(rce->newQScale);<br>
>>> - }<br>
>>> - if (expectedBits >= targetBits && step > 1)<br>
>>> - factor *= 0.90;<br>
>>> - else<br>
>>> - break;<br>
>>> - }<br>
>>> -<br>
>>> - if (m_isVbv && endIndex + fps < m_numEntries)<br>
>>> - if (!vbv2Pass((uint64_t)targetBits, endIndex + fps, endIndex + 1))<br>
>>> - return false;<br>
>>> -<br>
>>> - targetBits = 0;<br>
>>> - expectedBits = 0;<br>
>>> -<br>
>>> - for (int start = endIndex - fps + 1; start <= endIndex; start++)<br>
>>> - {<br>
>>> - RateControlEntry *rce = &m_rce2Pass[start];<br>
>>> - targetBits += qScale2bits(rce, x265_qp2qScale(rce->qpNoVbv));<br>
>>> - }<br>
>>> - while (1)<br>
>>> - {<br>
>>> - step = pow(2, factor / 6.0);<br>
>>> - expectedBits = 0;<br>
>>> - for (int start = endIndex - fps + 1; start <= endIndex; start++)<br>
>>> - {<br>
>>> - RateControlEntry *rce = &m_rce2Pass[start];<br>
>>> - rce->newQScale = rce->qScale * step;<br>
>>> - X265_CHECK(rce->newQScale >= 0, "new Qscale is negative\n");<br>
>>> - expectedBits += qScale2bits(rce, rce->newQScale);<br>
>>> - rce->newQp = x265_qScale2qp(rce->newQScale);<br>
>>> - }<br>
>>> - if (expectedBits > targetBits && step > 1)<br>
>>> - factor *= 1.1;<br>
>>> - else<br>
>>> - break;<br>
>>> - }<br>
>>> - if (m_isVbv)<br>
>>> - if (!vbv2Pass((uint64_t)targetBits, endIndex, endIndex - fps + 1))<br>
>>> - return false;<br>
>>> - diffQp = 0;<br>
>>> - m_reencode = endIndex - fps + 1;<br>
>>> - endIndex = endIndex + fps;<br>
>>> - startIndex = endIndex + 1;<br>
>>> - m_start = startIndex;<br>
>>> - targetBits = expectedBits = 0;<br>
>>> - }<br>
>>> - else<br>
>>> - targetBits = expectedBits = 0;<br>
>>> - }<br>
>>> - }<br>
>>> - else<br>
>>> - m_isQpModified = false;<br>
>>> - }<br>
>>> + allConstBits += m_rce2Pass[endIndex].miscBits;<br>
>>> + allCodedBits += m_rce2Pass[endIndex].coeffBits + m_rce2Pass[endIndex].mvBits;<br>
>>> }<br>
>>> - }<br>
>>><br>
>>> - if (m_param->rc.rateControlMode == X265_RC_ABR)<br>
>>> - {<br>
>>> if (allAvailableBits < allConstBits)<br>
>>> {<br>
>>> x265_log(m_param, X265_LOG_ERROR, "requested bitrate is too low. estimated minimum is %d kbps\n",<br>
>>> - (int)(allConstBits * m_fps / framesCount * 1000.));<br>
>>> + (int)(allConstBits * m_fps / framesCount * 1000.));<br>
>>> return false;<br>
>>> }<br>
>>> if (!analyseABR2Pass(allAvailableBits))<br>
>>> return false;<br>
>>> +<br>
>>> + return true;<br>
>>> }<br>
>>><br>
>>> - m_start = X265_MAX(m_start, endIndex - fps);<br>
>>> + if (m_param->rc.rateControlMode != X265_RC_CRF)<br>
>>> + {<br>
>>> + return true;<br>
>>> + }<br>
>>> +<br>
>>> + if (m_isQpModified)<br>
>>> + {<br>
>>> + return true;<br>
>>> + }<br>
>>> +<br>
>>> + if (m_start + (fps << 1) > m_numEntries)<br>
>>> + {<br>
>>> + return true;<br>
>>> + }<br>
>>> +<br>
>>> + for (startIndex = m_start, endIndex = m_numEntries - 1; startIndex < endIndex; startIndex++, endIndex--)<br>
>>> + {<br>
>>> + cpxSum += m_rce2Pass[startIndex].qScale / m_rce2Pass[startIndex].coeffBits;<br>
>>> + cpxSum2 += m_rce2Pass[endIndex].qScale / m_rce2Pass[endIndex].coeffBits;<br>
>>> +<br>
>>> + RateControlEntry *rce = &m_rce2Pass[startIndex];<br>
>>> + targetBits += qScale2bits(rce, x265_qp2qScale(rce->qpNoVbv));<br>
>>> + expectedBits += qScale2bits(rce, rce->qScale);<br>
>>> +<br>
>>> + rce = &m_rce2Pass[endIndex];<br>
>>> + targetBits2 += qScale2bits(rce, x265_qp2qScale(rce->qpNoVbv));<br>
>>> + expectedBits2 += qScale2bits(rce, rce->qScale);<br>
>>> + }<br>
>>> +<br>
>>> + if (expectedBits < 0.95 * targetBits || expectedBits2 < 0.95 * targetBits2)<br>
>>> + {<br>
>>> + if (cpxSum/cpxSum2 < 0.95 || cpxSum2 / cpxSum < 0.95)<br>
>>> + {<br>
>>> + m_isQpModified = true;<br>
>>> + m_isGopReEncoded = true;<br>
>>> +<br>
>>> + m_shortTermCplxSum = 0;<br>
>>> + m_shortTermCplxCount = 0;<br>
>>> + m_framesDone = m_start;<br>
>>> +<br>
>>> + for (startIndex = m_start; startIndex < m_numEntries; startIndex++)<br>
>>> + {<br>
>>> + m_shortTermCplxSum *= 0.5;<br>
>>> + m_shortTermCplxCount *= 0.5;<br>
>>> + m_shortTermCplxSum += m_rce2Pass[startIndex].currentSatd / (CLIP_DURATION(m_frameDuration) / BASE_FRAME_DURATION);<br>
>>> + m_shortTermCplxCount++;<br>
>>> + }<br>
>>> +<br>
>>> + m_bufferFill = m_rce2Pass[m_start - 1].bufferFill;<br>
>>> + m_bufferFillFinal = m_rce2Pass[m_start - 1].bufferFillFinal;<br>
>>> + m_bufferFillActual = m_rce2Pass[m_start - 1].bufferFillActual;<br>
>>> +<br>
>>> + m_reencode = m_start;<br>
>>> + m_start = m_numEntries;<br>
>>> + }<br>
>>> + else<br>
>>> + {<br>
>>> +<br>
>>> + m_isQpModified = false;<br>
>>> + m_isGopReEncoded = false;<br>
>>> + }<br>
>>> + }<br>
>>> + else<br>
>>> + {<br>
>>> +<br>
>>> + m_isQpModified = false;<br>
>>> + m_isGopReEncoded = false;<br>
>>> + }<br>
>>> +<br>
>>> + m_start = X265_MAX(m_start, m_numEntries - distance + m_param->keyframeMax);<br>
>>><br>
>>> return true;<br>
>>> }<br>
>>> @@ -1505,15 +1488,34 @@ int RateControl::rateControlStart(Frame* curFrame, RateControlEntry* rce, Encode<br>
>>> rce->frameSizeMaximum *= m_param->maxAUSizeFactor;<br>
>>> }<br>
>>> }<br>
>>> +<br>
>>> + ///< regenerate the qp<br>
>>> if (!m_isAbr && m_2pass && m_param->rc.rateControlMode == X265_RC_CRF)<br>
>>> {<br>
>>> - rce->qpPrev = x265_qScale2qp(rce->qScale);<br>
>>> - rce->qScale = rce->newQScale;<br>
>>> - rce->qpaRc = curEncData.m_avgQpRc = curEncData.m_avgQpAq = x265_qScale2qp(rce->newQScale);<br>
>>> - m_qp = int(rce->qpaRc + 0.5);<br>
>>> - rce->frameSizePlanned = qScale2bits(rce, rce->qScale);<br>
>>> - m_framesDone++;<br>
>>> - return m_qp;<br>
>>> + int index = m_encOrder[rce->poc];<br>
>>> + index++;<br>
>>> + double totalDuration = m_frameDuration;<br>
>>> + for (int j = 0; totalDuration < 1.0; j++)<br>
>>> + {<br>
>>> + switch (m_rce2Pass[index].sliceType)<br>
>>> + {<br>
>>> + case B_SLICE:<br>
>>> + curFrame->m_lowres.plannedType[j] = m_rce2Pass[index].keptAsRef ? X265_TYPE_BREF : X265_TYPE_B;<br>
>>> + break;<br>
>>> + case P_SLICE:<br>
>>> + curFrame->m_lowres.plannedType[j] = X265_TYPE_P;<br>
>>> + break;<br>
>>> + case I_SLICE:<br>
>>> + curFrame->m_lowres.plannedType[j] = m_param->bOpenGOP ? X265_TYPE_I : X265_TYPE_IDR;<br>
>>> + break;<br>
>>> + default:<br>
>>> + break;<br>
>>> + }<br>
>>> +<br>
>>> + curFrame->m_lowres.plannedSatd[j] = m_rce2Pass[index].currentSatd;<br>
>>> + totalDuration += m_frameDuration;<br>
>>> + index++;<br>
>>> + }<br>
>>> }<br>
>>><br>
>>> if (m_isAbr || m_2pass) // ABR,CRF<br>
>>> @@ -2019,7 +2021,7 @@ double RateControl::rateEstimateQscale(Frame* curFrame, RateControlEntry *rce)<br>
>>> qScale = x265_clip3(lqmin, lqmax, qScale);<br>
>>> }<br>
>>><br>
>>> - if (!m_2pass || m_param->bliveVBV2pass)<br>
>>> + if (!m_2pass || m_param->bliveVBV2pass || (m_2pass && m_param->rc.rateControlMode == X265_RC_CRF))<br>
>>> {<br>
>>> /* clip qp to permissible range after vbv-lookahead estimation to avoid possible<br>
>>> * mispredictions by initial frame size predictors */<br>
>>> @@ -2056,7 +2058,7 @@ double RateControl::rateEstimateQscale(Frame* curFrame, RateControlEntry *rce)<br>
>>> else<br>
>>> {<br>
>>> double abrBuffer = 2 * m_rateTolerance * m_bitrate;<br>
>>> - if (m_2pass)<br>
>>> + if (m_2pass && m_param->rc.rateControlMode != X265_RC_CRF)<br>
>>> {<br>
>>> double lmin = m_lmin[m_sliceType];<br>
>>> double lmax = m_lmax[m_sliceType];<br>
>>> @@ -2947,7 +2949,7 @@ int RateControl::rateControlEnd(Frame* curFrame, int64_t bits, RateControlEntry*<br>
>>><br>
>>> if (m_param->rc.aqMode || m_isVbv || m_param->bAQMotion || bEnableDistOffset)<br>
>>> {<br>
>>> - if (m_isVbv && !(m_2pass && m_param->rc.rateControlMode == X265_RC_CRF))<br>
>>> + if (m_isVbv)<br>
>>> {<br>
>>> double avgQpRc = 0;<br>
>>> /* determine avg QP decided by VBV rate control */<br>
>>> @@ -2981,16 +2983,7 @@ int RateControl::rateControlEnd(Frame* curFrame, int64_t bits, RateControlEntry*<br>
>>> if (m_param->rc.rateControlMode == X265_RC_CRF)<br>
>>> {<br>
>>> double crfVal, qpRef = curEncData.m_avgQpRc;<br>
>>> - bool is2passCrfChange = false;<br>
>>> - if (m_2pass)<br>
>>> - {<br>
>>> - if (fabs(curEncData.m_avgQpRc - rce->qpPrev) > 0.1)<br>
>>> - {<br>
>>> - qpRef = rce->qpPrev;<br>
>>> - is2passCrfChange = true;<br>
>>> - }<br>
>>> - }<br>
>>> - if (is2passCrfChange || fabs(qpRef - rce->qpNoVbv) > 0.5)<br>
>>> + if (fabs(qpRef - rce->qpNoVbv) > 0.5)<br>
>>> {<br>
>>> double crfFactor = rce->qRceq /x265_qp2qScale(qpRef);<br>
>>> double baseCplx = m_ncu * (m_param->bframes ? 120 : 80);<br>
>>> diff --git a/source/encoder/ratecontrol.h b/source/encoder/ratecontrol.h<br>
>>> index a0b555f8a..666025823 100644<br>
>>> --- a/source/encoder/ratecontrol.h<br>
>>> +++ b/source/encoder/ratecontrol.h<br>
>>> @@ -74,6 +74,7 @@ struct RateControlEntry<br>
>>> Predictor rowPreds[3][2];<br>
>>> Predictor* rowPred[2];<br>
>>><br>
>>> + int64_t currentSatd;<br>
>>> int64_t lastSatd; /* Contains the picture cost of the previous frame, required for resetAbr and VBV */<br>
>>> int64_t leadingNoBSatd;<br>
>>> int64_t rowTotalBits; /* update cplxrsum and totalbits at the end of 2 rows */<br>
>>> @@ -88,6 +89,8 @@ struct RateControlEntry<br>
>>> double rowCplxrSum;<br>
>>> double qpNoVbv;<br>
>>> double bufferFill;<br>
>>> + double bufferFillFinal;<br>
>>> + double bufferFillActual;<br>
>>> double targetFill;<br>
>>> bool vbvEndAdj;<br>
>>> double frameDuration;<br>
>>> --<br>
>>> 2.22.0.windows.1<br>
>>><br>
>>> _______________________________________________<br>
>>> x265-devel mailing list<br>
>>> <a href="mailto:x265-devel@videolan.org" target="_blank">x265-devel@videolan.org</a><br>
>>> <a href="https://mailman.videolan.org/listinfo/x265-devel" rel="noreferrer" target="_blank">https://mailman.videolan.org/listinfo/x265-devel</a><br>
>><br>
>> _______________________________________________<br>
>> x265-devel mailing list<br>
>> <a href="mailto:x265-devel@videolan.org" target="_blank">x265-devel@videolan.org</a><br>
>> <a href="https://mailman.videolan.org/listinfo/x265-devel" rel="noreferrer" target="_blank">https://mailman.videolan.org/listinfo/x265-devel</a><br>
><br>
> _______________________________________________<br>
> x265-devel mailing list<br>
> <a href="mailto:x265-devel@videolan.org" target="_blank">x265-devel@videolan.org</a><br>
> <a href="https://mailman.videolan.org/listinfo/x265-devel" rel="noreferrer" target="_blank">https://mailman.videolan.org/listinfo/x265-devel</a><br>
_______________________________________________<br>
x265-devel mailing list<br>
<a href="mailto:x265-devel@videolan.org" target="_blank">x265-devel@videolan.org</a><br>
<a href="https://mailman.videolan.org/listinfo/x265-devel" rel="noreferrer" target="_blank">https://mailman.videolan.org/listinfo/x265-devel</a><br>
</blockquote></div>