[x265] [PATCH 2 of 3] Implementation for Intra refresh
Deepthi Nandakumar
deepthi at multicorewareinc.com
Wed Sep 30 09:35:03 CEST 2015
So, some changes we discussed are
1. Remove extra field from x265_param. If the user calls the intra-refresh
API, set an Encoder field instead.
2. The scenecut algorithm shouldnt be touched by intra refresh, but intra
refresh should prevent the slicetype from changing to I on scenecut
detection. Print encoder warning about turning off scenecut.
3. There's an extra field in PIR (startCol, endCol, position) - this should
be trimmed down.
We also want to warn against dumb use of this feature - for instance, low
keyframeMax. Every frame is likely to be a refresh. The preferred use of
this feature is very high keyframeMax (which will cause a refresh anyway),
and an API call to trigger refresh when necessary.
On Tue, Sep 29, 2015 at 6:57 PM, Steve Borho <steve at borho.org> wrote:
> On 09/29, Santhoshini Sekar wrote:
> > On Mon, Sep 28, 2015 at 3:03 PM, Deepthi Nandakumar <
> > deepthi at multicorewareinc.com> wrote:
> >
> > > So, why should maxNumReferences be 1 always?
> > >
> > > I'm nicely confused by the requirement here. When intra-refresh is
> enabled
> > > (even if the user did not call the API function x265_intra_refresh),
> all I
> > > frames are converted to P frames. The responsibility of the API call is
> > > only to start a mid-GOP refresh. Does this require a no-scenecut, if
> not,
> > > what happens when a scenecut is inserted mid-GOP? Trigger a refresh?
> > >
> > > On Thu, Sep 24, 2015 at 9:00 AM, <santhoshini at multicorewareinc.com>
> wrote:
> > >
> > >> # HG changeset patch
> > >> # User Santhoshini Sekar<santhoshini at multicorewareinc.com>
> > >> # Date 1442393819 -19800
> > >> # Wed Sep 16 14:26:59 2015 +0530
> > >> # Node ID 97b62cb57bfa171e50d3fa736527634bb507cbe5
> > >> # Parent 98c0dcd5a10b8806aa1ceb775ff9342f7a7ae6c6
> > >> Implementation for Intra refresh
> > >>
> > >> diff -r 98c0dcd5a10b -r 97b62cb57bfa source/encoder/analysis.cpp
> > >> --- a/source/encoder/analysis.cpp Wed Sep 09 14:52:35 2015 +0530
> > >> +++ b/source/encoder/analysis.cpp Wed Sep 16 14:26:59 2015 +0530
> > >> @@ -171,10 +171,14 @@
> > >> }
> > >> else
> > >> {
> > >> - if (!m_param->rdLevel)
> > >> + if (m_param->bIntraRefresh && m_slice->m_sliceType ==
> P_SLICE &&
> > >> + ctu.m_cuPelX / g_maxCUSize >=
> > >> frame.m_encData->m_pir.pirStartCol
> > >> + && ctu.m_cuPelX / g_maxCUSize <
> > >> frame.m_encData->m_pir.pirEndCol)
> > >> + compressIntraCU(ctu, cuGeom, zOrder, qp);
> > >> + else if (!m_param->rdLevel)
> > >> {
> > >> /* In RD Level 0/1, copy source pixels into the
> > >> reconstructed block so
> > >> - * they are available for intra predictions */
> > >> + * they are available for intra predictions */
> > >> m_modeDepth[0].fencYuv.copyToPicYuv(*m_frame->m_reconPic,
> > >> ctu.m_cuAddr, 0);
> > >>
> > >> compressInterCU_rd0_4(ctu, cuGeom, qp);
> > >> @@ -1458,13 +1462,23 @@
> > >> bestPred->sa8dCost = MAX_INT64;
> > >> int bestSadCand = -1;
> > >> int sizeIdx = cuGeom.log2CUSize - 2;
> > >> -
> > >> + int safeX, maxSafeMv;
> > >> + if (m_param->bIntraRefresh && m_slice->m_sliceType == P_SLICE)
> > >> + {
> > >> + safeX =
> > >> m_slice->m_refFrameList[0][0]->m_encData->m_pir.pirEndCol *
> g_maxCUSize - 3;
> > >> + maxSafeMv = (safeX - tempPred->cu.m_cuPelX) * 4;
> > >> + }
> > >> for (uint32_t i = 0; i < numMergeCand; ++i)
> > >> {
> > >> if (m_bFrameParallel &&
> > >> (candMvField[i][0].mv.y >= (m_param->searchRange + 1) *
> 4 ||
> > >> candMvField[i][1].mv.y >= (m_param->searchRange + 1) *
> 4))
> > >> continue;
> > >> + if (m_param->bIntraRefresh && m_slice->m_sliceType ==
> P_SLICE &&
> > >> + tempPred->cu.m_cuPelX / g_maxCUSize <
> > >> m_frame->m_encData->m_pir.pirEndCol &&
> > >> + candMvField[i][0].mv.x > maxSafeMv)
> > >> + // skip merge candidates which reference beyond safe
> > >> reference area
> > >> + continue;
> > >>
> > >> tempPred->cu.m_mvpIdx[0][0] = (uint8_t)i; // merge candidate
> ID
> > >> is stored in L0 MVP idx
> > >> X265_CHECK(m_slice->m_sliceType == B_SLICE || !(candDir[i] &
> > >> 0x10), " invalid merge for P slice\n");
> > >> @@ -1570,7 +1584,12 @@
> > >> first = *m_reuseBestMergeCand;
> > >> last = first + 1;
> > >> }
> > >> -
> > >> + int safeX, maxSafeMv;
> > >> + if (m_param->bIntraRefresh && m_slice->m_sliceType == P_SLICE)
> > >> + {
> > >> + safeX =
> > >> m_slice->m_refFrameList[0][0]->m_encData->m_pir.pirEndCol *
> g_maxCUSize - 3;
> > >> + maxSafeMv = (safeX - tempPred->cu.m_cuPelX) * 4;
> > >> + }
> > >> for (uint32_t i = first; i < last; i++)
> > >> {
> > >> if (m_bFrameParallel &&
> > >> @@ -1593,7 +1612,11 @@
> > >> continue;
> > >> triedBZero = true;
> > >> }
> > >> -
> > >> + if (m_param->bIntraRefresh && m_slice->m_sliceType ==
> P_SLICE &&
> > >> + tempPred->cu.m_cuPelX / g_maxCUSize <
> > >> m_frame->m_encData->m_pir.pirEndCol &&
> > >> + candMvField[i][0].mv.x > maxSafeMv)
> > >> + // skip merge candidates which reference beyond safe
> > >> reference area
> > >> + continue;
> > >> tempPred->cu.m_mvpIdx[0][0] = (uint8_t)i; /* merge
> candidate
> > >> ID is stored in L0 MVP idx */
> > >> tempPred->cu.m_interDir[0] = candDir[i];
> > >> tempPred->cu.m_mv[0][0] = candMvField[i][0].mv;
> > >> diff -r 98c0dcd5a10b -r 97b62cb57bfa source/encoder/api.cpp
> > >> --- a/source/encoder/api.cpp Wed Sep 09 14:52:35 2015 +0530
> > >> +++ b/source/encoder/api.cpp Wed Sep 16 14:26:59 2015 +0530
> > >> @@ -245,6 +245,16 @@
> > >> }
> > >> }
> > >>
> > >> +int x265_encoder_intra_refresh(x265_encoder *enc)
> > >> +{
> > >> + if (!enc)
> > >> + return -1;
> > >> +
> > >> + Encoder *encoder = static_cast<Encoder*>(enc);
> > >> + encoder->m_param->bQueuedIntraRefresh = 1;
> > >> + return 0;
> > >> +}
> > >> +
> > >> void x265_cleanup(void)
> > >> {
> > >> if (!g_ctuSizeConfigured)
> > >> @@ -317,6 +327,7 @@
> > >> &x265_cleanup,
> > >>
> > >> sizeof(x265_frame_stats),
> > >> + &x265_encoder_intra_refresh,
> > >> };
> > >>
> > >> typedef const x265_api* (*api_get_func)(int bitDepth);
> > >> diff -r 98c0dcd5a10b -r 97b62cb57bfa source/encoder/encoder.cpp
> > >> --- a/source/encoder/encoder.cpp Wed Sep 09 14:52:35 2015 +0530
> > >> +++ b/source/encoder/encoder.cpp Wed Sep 16 14:26:59 2015 +0530
> > >> @@ -437,6 +437,46 @@
> > >> }
> > >> }
> > >>
> > >> +void Encoder::calcRefreshInterval(Frame* frameEnc)
> > >> +{
> > >> + Slice* slice = frameEnc->m_encData->m_slice;
> > >> + uint32_t numBlocksInRow = slice->m_sps->numCuInWidth;
> > >> + FrameData::PeriodicIR* pir = &frameEnc->m_encData->m_pir;
> > >> + if (slice->m_sliceType == I_SLICE)
> > >> + {
> > >> + pir->framesSinceLastPir = 0;
> > >> + m_param->bQueuedIntraRefresh = 0;
> > >> + /* PIR is currently only supported with ref == 1, so any
> intra
> > >> frame effectively refreshes
> > >> + * the whole frame and counts as an intra refresh. */
> > >> + pir->position = numBlocksInRow;
> > >> + }
> > >> + else if (slice->m_sliceType == P_SLICE)
> > >> + {
> > >> + Frame* ref =
> frameEnc->m_encData->m_slice->m_refFrameList[0][0];
> > >> + int pocdiff = frameEnc->m_poc - ref->m_poc;
> > >> + float increment = X265_MAX(((float)numBlocksInRow - 1) /
> > >> m_param->keyframeMax, 1);
> > >> + pir->position = ref->m_encData->m_pir.position;
> > >> + pir->framesSinceLastPir =
> > >> ref->m_encData->m_pir.framesSinceLastPir + pocdiff;
> > >> + if (pir->framesSinceLastPir >= m_param->keyframeMax ||
> > >> + (m_param->bQueuedIntraRefresh && pir->position + 0.5 >=
> > >> numBlocksInRow))
> > >> + {
> > >> + pir->position = 0;
> > >> + pir->framesSinceLastPir = 0;
> > >> + m_param->bQueuedIntraRefresh = 0;
> > >> + frameEnc->m_lowres.bKeyframe = 1;
> > >> + }
> > >> + pir->pirStartCol = (uint32_t)(pir->position + 0.5);
> > >> + pir->position += increment * pocdiff;
> > >> + pir->pirEndCol = (uint32_t)(pir->position + 0.5);
> > >> + /* If our intra refresh has reached the right side of the
> frame,
> > >> we're done. */
> > >> + if (pir->pirEndCol >= numBlocksInRow)
> > >> + {
> > >> + pir->position = numBlocksInRow;
> > >> + pir->pirEndCol = numBlocksInRow;
> > >> + }
> > >> + }
> > >> +}
> > >> +
> > >> /**
> > >> * Feed one new input frame into the encoder, get one frame out. If
> > >> pic_in is
> > >> * NULL, a flush condition is implied and pic_in must be NULL for all
> > >> subsequent
> > >> @@ -768,6 +808,8 @@
> > >>
> > >> if (m_param->rc.rateControlMode != X265_RC_CQP)
> > >> m_lookahead->getEstimatedPictureCost(frameEnc);
> > >> + if (m_param->bIntraRefresh)
> > >> + calcRefreshInterval(frameEnc);
> > >>
> > >> /* Allow FrameEncoder::compressFrame() to start in the
> frame
> > >> encoder thread */
> > >> if (!curEncoder->startCompressFrame(frameEnc))
> > >> diff -r 98c0dcd5a10b -r 97b62cb57bfa source/encoder/encoder.h
> > >> --- a/source/encoder/encoder.h Wed Sep 09 14:52:35 2015 +0530
> > >> +++ b/source/encoder/encoder.h Wed Sep 16 14:26:59 2015 +0530
> > >> @@ -168,6 +168,8 @@
> > >>
> > >> void finishFrameStats(Frame* pic, FrameEncoder *curEncoder,
> uint64_t
> > >> bits, x265_frame_stats* frameStats);
> > >>
> > >> + void calcRefreshInterval(Frame* frameEnc);
> > >> +
> > >> protected:
> > >>
> > >> void initVPS(VPS *vps);
> > >> diff -r 98c0dcd5a10b -r 97b62cb57bfa source/encoder/search.cpp
> > >> --- a/source/encoder/search.cpp Wed Sep 09 14:52:35 2015 +0530
> > >> +++ b/source/encoder/search.cpp Wed Sep 16 14:26:59 2015 +0530
> > >> @@ -2460,6 +2460,17 @@
> > >> cu.clipMv(mvmin);
> > >> cu.clipMv(mvmax);
> > >>
> > >> + if (cu.m_encData->m_param->bIntraRefresh && m_slice->m_sliceType
> ==
> > >> P_SLICE &&
> > >> + cu.m_cuPelX / g_maxCUSize <
> > >> m_frame->m_encData->m_pir.pirStartCol &&
> > >> + m_slice->m_refFrameList[0][0]->m_encData->m_pir.pirEndCol <
> > >> m_slice->m_sps->numCuInWidth)
> > >>
> > >
> > > Why check for
> m_slice->m_refFrameList[0][0]->m_encData->m_pir.pirEndCol <
> > > m_slice->m_sps->numCuInWidth here? This should be taken care of in
> > > calcRefreshInterval.
> > >
> >
> > In calcRefreshInterval, pirEndCol is set with value which indicates the
> > maximum edge of the safe region for reference in intra refresh. Once we
> > reach the right side of the frame, we signal it in
> > pirEndCol by setting it with numCuInWidth. Indicating that we can no
> longer
> > use this for reference during intra refresh. So in places where we set
> max
> > and min mv range, it is necessary to check this condition. We anyway need
> > another flag if we want this check in calcRefreshInterval.
>
> More than that, if the reference frame was not part of a refresh sweep,
> we do not want to stop motion searches at the right edge. There is a
> difference between limiting the ME range to the right picture edge and
> having no limit at all (aka, the right extended border). Searches need
> to be able to cross the picture edge
>
> --
> Steve Borho
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20150930/69defd5f/attachment-0001.html>
More information about the x265-devel
mailing list