[x265] [PATCH] Add real-time VBV fullness based QP tuning in VBV 2 pass
Kirithika Kalirathnam
kirithika at multicorewareinc.com
Wed Sep 9 07:22:13 CEST 2020
On Mon, Sep 7, 2020 at 6:19 PM Aruna Matheswaran <aruna at multicorewareinc.com>
wrote:
>
>
> On Fri, Sep 4, 2020 at 5:44 PM Kirithika Kalirathnam <
> kirithika at multicorewareinc.com> wrote:
>
>> From f1391f094c9c3c0502624532f751219d126905aa Mon Sep 17 00:00:00 2001
>> From: Kirithika <kirithika at multicorewareinc.com>
>> Date: Fri, 28 Aug 2020 09:20:47 +0530
>> Subject: [PATCH] Add real-time VBV fullness based QP tuning in VBV 2 pass
>>
>> This commit enables real-time VBV fullness based 2nd pass QP tuning.
>> Experimental feature.Default disabled
>> ---
>> doc/reST/cli.rst | 9 +++
>> source/CMakeLists.txt | 2 +-
>> source/common/param.cpp | 12 ++++
>> source/encoder/ratecontrol.cpp | 100 ++++++++++++++++++---------------
>> source/encoder/slicetype.cpp | 14 +++--
>> source/x265.h | 4 ++
>> source/x265cli.cpp | 1 +
>> source/x265cli.h | 2 +
>> 8 files changed, 92 insertions(+), 52 deletions(-)
>>
>> diff --git a/doc/reST/cli.rst b/doc/reST/cli.rst
>> index e6c628c36..2a4b25aca 100755
>> --- a/doc/reST/cli.rst
>> +++ b/doc/reST/cli.rst
>> @@ -1977,6 +1977,15 @@ Quality, rate control and rate distortion options
>>
>> **Range of values:** 0 to 10
>>
>> +.. option:: --realtime-vbv-2pass, --no-realtime-vbv-2pass
>>
> [AM] Can we find a better alternate for the CLI option?
> --real-time-vbv-2-pass looks a bit long.
> The term 2pass is a little confusing. Technically it is multi-pass; not
> just 2pass.
>
Ok.I shall update the name to vbv-live-multi-pass,as it is self-explanatory
though it's a bit longer.
> +
>> + It enables the Qp tuning at frame level based on real time VBV Buffer
>> fullness
>> + in the ratecontrol second pass to reduce the VBV violations.
>> + It could only be enabled with rate control pass 2 encodes with VBV
>> and ABR
>> + rate control mode.
>> +
>>
> [AM] why not for CRF rate-control mode?
>
Currently,we have tested the feature only for ABR case.We shall extend this
to CRF mode also in future.
> + Default disabled. **Experimental feature**
>> +
>> Quantization Options
>> ====================
>>
>> diff --git a/source/CMakeLists.txt b/source/CMakeLists.txt
>> index d9ed3983e..5738c861c 100755
>> --- a/source/CMakeLists.txt
>> +++ b/source/CMakeLists.txt
>> @@ -29,7 +29,7 @@ option(NATIVE_BUILD "Target the build CPU" OFF)
>> option(STATIC_LINK_CRT "Statically link C runtime for release builds"
>> OFF)
>> mark_as_advanced(FPROFILE_USE FPROFILE_GENERATE NATIVE_BUILD)
>> # X265_BUILD must be incremented each time the public API is changed
>> -set(X265_BUILD 196)
>> +set(X265_BUILD 197)
>> configure_file("${PROJECT_SOURCE_DIR}/x265.def.in"
>> "${PROJECT_BINARY_DIR}/x265.def")
>> configure_file("${PROJECT_SOURCE_DIR}/x265_config.h.in"
>> diff --git a/source/common/param.cpp b/source/common/param.cpp
>> index 845c38bd2..fc0425905 100755
>> --- a/source/common/param.cpp
>> +++ b/source/common/param.cpp
>> @@ -288,6 +288,7 @@ void x265_param_default(x265_param* param)
>> param->bResetZoneConfig = 1;
>> param->reconfigWindowSize = 0;
>> param->decoderVbvMaxRate = 0;
>> + param->rc.bRealtimeVBV2pass = 0;
>>
>> /* Video Usability Information (VUI) */
>> param->vui.aspectRatioIdc = 0;
>> @@ -1375,6 +1376,7 @@ int x265_param_parse(x265_param* p, const char*
>> name, const char* value)
>> sscanf(value, "%d,%d,%d", &p->hmeRange[0], &p->hmeRange[1],
>> &p->hmeRange[2]);
>> p->bEnableHME = true;
>> }
>> + OPT("realtime-vbv-2pass") p->rc.bRealtimeVBV2pass =
>> atobool(value);
>> else
>> return X265_PARAM_BAD_NAME;
>> }
>> @@ -1817,6 +1819,15 @@ int x265_check_params(x265_param* param)
>> CHECK(param->confWinRightOffset < 0, "Conformance Window Right
>> Offset must be 0 or greater");
>> CHECK(param->confWinBottomOffset < 0, "Conformance Window Bottom
>> Offset must be 0 or greater");
>> CHECK(param->decoderVbvMaxRate < 0, "Invalid Decoder Vbv Maxrate.
>> Value can not be less than zero");
>> + if (param->rc.bRealtimeVBV2pass)
>> + {
>> + CHECK((param->rc.bStatRead == 0), "Real time VBV in 2 pass
>> option requires rate control 2 pass to be enabled");
>> + if ((param->rc.vbvMaxBitrate <= 0 || param->rc.vbvBufferSize <=
>> 0))
>> + {
>> + param->rc.bRealtimeVBV2pass = 0;
>> + x265_log(param, X265_LOG_WARNING, "Real time VBV enabled
>> without VBV settings.Disabling real time VBV in 2 pass\n");
>> + }
>> + }
>> return check_failed;
>> }
>>
>> @@ -2593,6 +2604,7 @@ void x265_copy_params(x265_param* dst, x265_param*
>> src)
>>
>> dst->confWinRightOffset = src->confWinRightOffset;
>> dst->confWinBottomOffset = src->confWinBottomOffset;
>> + dst->rc.bRealtimeVBV2pass = src->rc.bRealtimeVBV2pass;
>> #ifdef SVT_HEVC
>> memcpy(dst->svtHevcParam, src->svtHevcParam,
>> sizeof(EB_H265_ENC_CONFIGURATION));
>> #endif
>> diff --git a/source/encoder/ratecontrol.cpp
>> b/source/encoder/ratecontrol.cpp
>> index 82d7c4f2a..ebf374759 100644
>> --- a/source/encoder/ratecontrol.cpp
>> +++ b/source/encoder/ratecontrol.cpp
>> @@ -1755,34 +1755,32 @@ double RateControl::rateEstimateQscale(Frame*
>> curFrame, RateControlEntry *rce)
>> g_sliceTypeToChar[m_sliceType],
>> g_sliceTypeToChar[rce->sliceType]);
>> }
>> }
>> - else
>> +
>> + if ((m_param->rc.bRealtimeVBV2pass && m_param->rc.rateControlMode ==
>> X265_RC_ABR) || m_isAbr)
>> {
>> - if (m_isAbr)
>> + int pos = m_sliderPos % s_slidingWindowFrames;
>> + int addPos = (pos + s_slidingWindowFrames - 1) %
>> s_slidingWindowFrames;
>> + if (m_sliderPos > s_slidingWindowFrames)
>> {
>> - int pos = m_sliderPos % s_slidingWindowFrames;
>> - int addPos = (pos + s_slidingWindowFrames - 1) %
>> s_slidingWindowFrames;
>> - if (m_sliderPos > s_slidingWindowFrames)
>> - {
>> - const static double base = pow(0.5,
>> s_slidingWindowFrames - 1);
>> - m_movingAvgSum -= m_lastRemovedSatdCost * base;
>> - m_movingAvgSum *= 0.5;
>> - m_movingAvgSum += m_satdCostWindow[addPos];
>> - }
>> - else if (m_sliderPos == s_slidingWindowFrames)
>> - {
>> - m_movingAvgSum += m_satdCostWindow[addPos];
>> - }
>> - else if (m_sliderPos > 0)
>> - {
>> - m_movingAvgSum += m_satdCostWindow[addPos];
>> - m_movingAvgSum *= 0.5;
>> - }
>> -
>> - rce->movingAvgSum = m_movingAvgSum;
>> - m_lastRemovedSatdCost = m_satdCostWindow[pos];
>> - m_satdCostWindow[pos] = rce->lastSatd;
>> - m_sliderPos++;
>> + const static double base = pow(0.5, s_slidingWindowFrames -
>> 1);
>> + m_movingAvgSum -= m_lastRemovedSatdCost * base;
>> + m_movingAvgSum *= 0.5;
>> + m_movingAvgSum += m_satdCostWindow[addPos];
>> + }
>> + else if (m_sliderPos == s_slidingWindowFrames)
>> + {
>> + m_movingAvgSum += m_satdCostWindow[addPos];
>> }
>> + else if (m_sliderPos > 0)
>> + {
>> + m_movingAvgSum += m_satdCostWindow[addPos];
>> + m_movingAvgSum *= 0.5;
>> + }
>> +
>> + rce->movingAvgSum = m_movingAvgSum;
>> + m_lastRemovedSatdCost = m_satdCostWindow[pos];
>> + m_satdCostWindow[pos] = rce->lastSatd;
>> + m_sliderPos++;
>> }
>>
>> if (m_sliceType == B_SLICE)
>> @@ -1887,7 +1885,7 @@ double RateControl::rateEstimateQscale(Frame*
>> curFrame, RateControlEntry *rce)
>> qScale = x265_clip3(lqmin, lqmax, qScale);
>> }
>>
>> - if (!m_2pass)
>> + if (!m_2pass || m_param->rc.bRealtimeVBV2pass)
>> {
>> /* clip qp to permissible range after vbv-lookahead
>> estimation to avoid possible
>> * mispredictions by initial frame size predictors */
>> @@ -1955,7 +1953,9 @@ double RateControl::rateEstimateQscale(Frame*
>> curFrame, RateControlEntry *rce)
>> else
>> q /= zone->bitrateFactor;
>> }
>> - q /= x265_clip3(0.5, 2.0, (double)(abrBuffer - diff) /
>> abrBuffer);
>> + /*Existing ABR conformance check may not be valid with real
>> time VBV*/
>> + if(!m_param->rc.bRealtimeVBV2pass)
>> + q /= x265_clip3(0.5, 2.0, (double)(abrBuffer - diff) /
>> abrBuffer);
>> if (m_expectedBitsSum > 0)
>> {
>> /* Adjust quant based on the difference between
>> @@ -1988,24 +1988,34 @@ double RateControl::rateEstimateQscale(Frame*
>> curFrame, RateControlEntry *rce)
>>
>> if (m_isVbv)
>> {
>> - /* Do not overflow vbv */
>> - double expectedSize = qScale2bits(rce, q);
>> - double expectedVbv = m_bufferFill + m_bufferRate -
>> expectedSize;
>> - double expectedFullness = rce->expectedVbv /
>> m_bufferSize;
>> - double qmax = q * (2 - expectedFullness);
>> - double sizeConstraint = 1 + expectedFullness;
>> - qmax = X265_MAX(qmax, rce->newQScale);
>> - if (expectedFullness < .05)
>> - qmax = lmax;
>> - qmax = X265_MIN(qmax, lmax);
>> - while (((expectedVbv < rce->expectedVbv/sizeConstraint)
>> && (q < qmax)) ||
>> + if (!m_param->rc.bRealtimeVBV2pass)
>> + {
>> + /* Do not overflow vbv */
>> + double expectedSize = qScale2bits(rce, q);
>> + double expectedVbv = m_bufferFill + m_bufferRate -
>> expectedSize;
>> + double expectedFullness = rce->expectedVbv /
>> m_bufferSize;
>> + double qmax = q * (2 - expectedFullness);
>> + double sizeConstraint = 1 + expectedFullness;
>> + qmax = X265_MAX(qmax, rce->newQScale);
>> + if (expectedFullness < .05)
>> + qmax = lmax;
>> + qmax = X265_MIN(qmax, lmax);
>> + while (((expectedVbv < rce->expectedVbv /
>> sizeConstraint) && (q < qmax)) ||
>> ((expectedVbv < 0) && (q < lmax)))
>> + {
>> + q *= 1.05;
>> + expectedSize = qScale2bits(rce, q);
>> + expectedVbv = m_bufferFill + m_bufferRate -
>> expectedSize;
>> + }
>> + }
>> + else
>> {
>> - q *= 1.05;
>> - expectedSize = qScale2bits(rce, q);
>> - expectedVbv = m_bufferFill + m_bufferRate -
>> expectedSize;
>> + /* clip qp to permissible range after
>> vbv-lookahead estimation to avoid possible
>> + * mispredictions by Rate Control pass 1
>> statistics analysis */
>> + q = clipQscale(curFrame, rce, q);
>> }
>> }
>> +
>> q = x265_clip3(lmin, lmax, q);
>> }
>> else
>> @@ -2301,7 +2311,7 @@ double RateControl::clipQscale(Frame* curFrame,
>> RateControlEntry* rce, double q)
>> double lmin = m_lmin[rce->sliceType];
>> double lmax = m_lmax[rce->sliceType];
>> double q0 = q;
>> - if (m_isVbv && m_currentSatd > 0 && curFrame)
>> + if ((m_isVbv || m_param->rc.bRealtimeVBV2pass) && curFrame &&
>> m_currentSatd > 0 )
>> {
>>
> [AM] Do we need to m_param->rc.bRealtimeVBV2pass check here? The call to
> clipQscale() is guarded by "m_param->rc.bRealtimeVBV2pass" from what I see.
>
yeah,the check is redundant and I shall remove it.
> if (m_param->lookaheadDepth || m_param->rc.cuTree ||
>> (m_param->scenecutThreshold || m_param->bHistBasedSceneCut)
>> ||
>> @@ -2379,7 +2389,7 @@ double RateControl::clipQscale(Frame* curFrame,
>> RateControlEntry* rce, double q)
>> }
>> /* Try to get the buffer not more than 80% filled,
>> but don't set an impossible goal. */
>> targetFill = x265_clip3(m_bufferSize * (1 - 0.2 *
>> finalDur), m_bufferSize, m_bufferFill - totalDuration * m_vbvMaxRate * 0.5);
>> - if (m_isCbr && bufferFillCur > targetFill &&
>> !m_isSceneTransition)
>> + if ((m_isCbr || m_2pass) && bufferFillCur >
>> targetFill && !m_isSceneTransition)
>> {
>> q /= 1.01;
>> loopTerminate |= 2;
>> @@ -2432,7 +2442,7 @@ double RateControl::clipQscale(Frame* curFrame,
>> RateControlEntry* rce, double q)
>> * lookahead vbv reduces its qscale by half its value. Be on
>> safer side and avoid drastic
>> * qscale reductions for frames high in complexity */
>> bool mispredCheck = rce->movingAvgSum && m_currentSatd >=
>> rce->movingAvgSum && q <= q0 / 2;
>> - if (!m_isCbr || (m_isAbr && mispredCheck))
>> + if (!m_isCbr || ((m_isAbr || m_2pass) && mispredCheck))
>>
> [AM] Same question here.
>
m_isAbr is not set with 2 pass encodes.Hence I have added an additional
check here.
> q = X265_MAX(q0, q);
>>
>> if (m_rateFactorMaxIncrement)
>> @@ -2442,7 +2452,7 @@ double RateControl::clipQscale(Frame* curFrame,
>> RateControlEntry* rce, double q)
>> return x265_clip3(lmin, qmax, q);
>> }
>> }
>> - if (m_2pass)
>> + if (!curFrame && m_2pass)
>> {
>> double min = log(lmin);
>> double max = log(lmax);
>> diff --git a/source/encoder/slicetype.cpp b/source/encoder/slicetype.cpp
>> index 4e52e584c..69f3b0ad1 100644
>> --- a/source/encoder/slicetype.cpp
>> +++ b/source/encoder/slicetype.cpp
>> @@ -1497,14 +1497,15 @@ void Lookahead::slicetypeDecide()
>> }
>> }
>>
>> - if (m_lastNonB && !m_param->rc.bStatRead &&
>> + if (m_lastNonB &&
>> ((m_param->bFrameAdaptive && m_param->bframes) ||
>> m_param->rc.cuTree || m_param->scenecutThreshold ||
>> m_param->bHistBasedSceneCut ||
>> (m_param->lookaheadDepth && m_param->rc.vbvBufferSize)))
>> {
>> - slicetypeAnalyse(frames, false);
>> + if(!m_param->rc.bStatRead)
>> + slicetypeAnalyse(frames, false);
>> bool bIsVbv = m_param->rc.vbvBufferSize > 0 &&
>> m_param->rc.vbvMaxBitrate > 0;
>> - if (m_param->analysisLoad && m_param->scaleFactor && bIsVbv)
>> + if ((m_param->analysisLoad && m_param->scaleFactor && bIsVbv) ||
>> m_param->rc.bRealtimeVBV2pass)
>> {
>> int numFrames;
>> for (numFrames = 0; numFrames < maxSearch; numFrames++)
>> @@ -1749,7 +1750,7 @@ void Lookahead::slicetypeDecide()
>> }
>> }
>>
>> - bool isKeyFrameAnalyse = (m_param->rc.cuTree ||
>> (m_param->rc.vbvBufferSize && m_param->lookaheadDepth)) &&
>> !m_param->rc.bStatRead;
>> + bool isKeyFrameAnalyse = (m_param->rc.cuTree ||
>> (m_param->rc.vbvBufferSize && m_param->lookaheadDepth));
>> if (isKeyFrameAnalyse && IS_X265_TYPE_I(m_lastNonB->sliceType))
>> {
>> m_inputLock.acquire();
>> @@ -1764,9 +1765,10 @@ void Lookahead::slicetypeDecide()
>> m_inputLock.release();
>>
>> frames[j + 1] = NULL;
>> - slicetypeAnalyse(frames, true);
>> + if (!m_param->rc.bStatRead)
>> + slicetypeAnalyse(frames, true);
>> bool bIsVbv = m_param->rc.vbvBufferSize > 0 &&
>> m_param->rc.vbvMaxBitrate > 0;
>> - if (m_param->analysisLoad && m_param->scaleFactor && bIsVbv)
>> + if ((m_param->analysisLoad && m_param->scaleFactor && bIsVbv) ||
>> m_param->rc.bRealtimeVBV2pass)
>> {
>> int numFrames;
>> for (numFrames = 0; numFrames < maxSearch; numFrames++)
>> diff --git a/source/x265.h b/source/x265.h
>> index 0ffa600b0..bcd1d4e58 100644
>> --- a/source/x265.h
>> +++ b/source/x265.h
>> @@ -1486,6 +1486,10 @@ typedef struct x265_param
>> /* internally enable if tune grain is set */
>> int bEnableConstVbv;
>>
>> + /*Enables Qp tuning with respect to real time VBV buffer
>> fullness in rate
>> + control 2 pass. Experimental.Default is disabled*/
>> + int bRealtimeVBV2pass;
>> +
>>
> [AM] Always add new param options to the end of the structure. Nested
> structures shall break backward compatibility.
>
Ok.I will add the new param at the end of param struct.
> } rc;
>>
>> /*== Video Usability Information ==*/
>> diff --git a/source/x265cli.cpp b/source/x265cli.cpp
>> index 6e8e0e661..0b17a8d3a 100755
>> --- a/source/x265cli.cpp
>> +++ b/source/x265cli.cpp
>> @@ -221,6 +221,7 @@ namespace X265_NS {
>> " - 3 : Nth pass,
>> overwrites stats file\n");
>> H0(" --[no-]multi-pass-opt-analysis Refine analysis in 2
>> pass based on analysis information from pass 1\n");
>> H0(" --[no-]multi-pass-opt-distortion Use distortion of CTU
>> from pass 1 to refine qp in 2 pass\n");
>> + H0(" --[no-]realtime-vbv-2pass Enable realtime VBV in rate
>> control 2 pass.Default %s\n", OPT(param->bLossless));
>> H0(" --stats Filename for stats file in
>> multipass pass rate control. Default x265_2pass.log\n");
>> H0(" --[no-]analyze-src-pics Motion estimation uses
>> source frame planes. Default disable\n");
>> H0(" --[no-]slow-firstpass Enable a slow first pass in
>> a multipass rate control mode. Default %s\n",
>> OPT(param->rc.bEnableSlowFirstPass));
>> diff --git a/source/x265cli.h b/source/x265cli.h
>> index 311f06935..89840d9af 100644
>> --- a/source/x265cli.h
>> +++ b/source/x265cli.h
>> @@ -284,6 +284,8 @@ static const struct option long_options[] =
>> { "no-multi-pass-opt-analysis", no_argument, NULL, 0 },
>> { "multi-pass-opt-distortion", no_argument, NULL, 0 },
>> { "no-multi-pass-opt-distortion", no_argument, NULL, 0 },
>> + { "realtime-vbv-2pass", no_argument, NULL, 0 },
>> + { "no-realtime-vbv-2pass", no_argument, NULL, 0 },
>>
> [AM] Please add an entry for the new option in x265_param2string(); This
> will keep the info SEI updated.
>
Ok,will update
> { "slow-firstpass", no_argument, NULL, 0 },
>> { "no-slow-firstpass", no_argument, NULL, 0 },
>> { "multi-pass-opt-rps", no_argument, NULL, 0 },
>> --
>> 2.28.0.windows.1
>>
>> *Thanks,*
>> *Kirithika*
>> _______________________________________________
>> x265-devel mailing list
>> x265-devel at videolan.org
>> https://mailman.videolan.org/listinfo/x265-devel
>>
>
>
> --
> Regards,
> *Aruna Matheswaran,*
> Video Codec Engineer,
> Media & AI analytics BU,
>
>
>
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20200909/8cc7d9f3/attachment-0001.html>
More information about the x265-devel
mailing list