<div dir="ltr"><div><div><div><div>So, some changes we discussed are<br><br></div>1. Remove extra field from x265_param. If the user calls the intra-refresh API, set an Encoder field instead. <br></div><br></div><div>2. The scenecut algorithm shouldnt be touched by intra refresh, but intra refresh should prevent the slicetype from changing to I on scenecut detection. Print encoder warning about turning off scenecut. <br></div><div><br></div>3. There's an extra field in PIR (startCol, endCol, position) - this should be trimmed down.<br><br></div>We also want to warn against dumb use of this feature - for instance, low keyframeMax. Every frame is likely to be a refresh. The preferred use of this feature is very high keyframeMax (which will cause a refresh anyway), and an API call to trigger refresh when necessary. <br></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Sep 29, 2015 at 6:57 PM, Steve Borho <span dir="ltr"><<a href="mailto:steve@borho.org" target="_blank">steve@borho.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On 09/29, Santhoshini Sekar wrote:<br>
> On Mon, Sep 28, 2015 at 3:03 PM, Deepthi Nandakumar <<br>
> <a href="mailto:deepthi@multicorewareinc.com">deepthi@multicorewareinc.com</a>> wrote:<br>
><br>
> > So, why should maxNumReferences be 1 always?<br>
> ><br>
> > I'm nicely confused by the requirement here. When intra-refresh is enabled<br>
> > (even if the user did not call the API function x265_intra_refresh), all I<br>
> > frames are converted to P frames. The responsibility of the API call is<br>
> > only to start a mid-GOP refresh. Does this require a no-scenecut, if not,<br>
> > what happens when a scenecut is inserted mid-GOP? Trigger a refresh?<br>
> ><br>
> > On Thu, Sep 24, 2015 at 9:00 AM, <<a href="mailto:santhoshini@multicorewareinc.com">santhoshini@multicorewareinc.com</a>> wrote:<br>
> ><br>
> >> # HG changeset patch<br>
> >> # User Santhoshini Sekar<<a href="mailto:santhoshini@multicorewareinc.com">santhoshini@multicorewareinc.com</a>><br>
> >> # Date 1442393819 -19800<br>
> >> # Wed Sep 16 14:26:59 2015 +0530<br>
> >> # Node ID 97b62cb57bfa171e50d3fa736527634bb507cbe5<br>
> >> # Parent 98c0dcd5a10b8806aa1ceb775ff9342f7a7ae6c6<br>
> >> Implementation for Intra refresh<br>
> >><br>
> >> diff -r 98c0dcd5a10b -r 97b62cb57bfa source/encoder/analysis.cpp<br>
> >> --- a/source/encoder/analysis.cpp Wed Sep 09 14:52:35 2015 +0530<br>
> >> +++ b/source/encoder/analysis.cpp Wed Sep 16 14:26:59 2015 +0530<br>
> >> @@ -171,10 +171,14 @@<br>
> >> }<br>
> >> else<br>
> >> {<br>
> >> - if (!m_param->rdLevel)<br>
> >> + if (m_param->bIntraRefresh && m_slice->m_sliceType == P_SLICE &&<br>
> >> + ctu.m_cuPelX / g_maxCUSize >=<br>
> >> frame.m_encData->m_pir.pirStartCol<br>
> >> + && ctu.m_cuPelX / g_maxCUSize <<br>
> >> frame.m_encData->m_pir.pirEndCol)<br>
> >> + compressIntraCU(ctu, cuGeom, zOrder, qp);<br>
> >> + else if (!m_param->rdLevel)<br>
> >> {<br>
> >> /* In RD Level 0/1, copy source pixels into the<br>
> >> reconstructed block so<br>
> >> - * they are available for intra predictions */<br>
> >> + * they are available for intra predictions */<br>
> >> m_modeDepth[0].fencYuv.copyToPicYuv(*m_frame->m_reconPic,<br>
> >> ctu.m_cuAddr, 0);<br>
> >><br>
> >> compressInterCU_rd0_4(ctu, cuGeom, qp);<br>
> >> @@ -1458,13 +1462,23 @@<br>
> >> bestPred->sa8dCost = MAX_INT64;<br>
> >> int bestSadCand = -1;<br>
> >> int sizeIdx = cuGeom.log2CUSize - 2;<br>
> >> -<br>
> >> + int safeX, maxSafeMv;<br>
> >> + if (m_param->bIntraRefresh && m_slice->m_sliceType == P_SLICE)<br>
> >> + {<br>
> >> + safeX =<br>
> >> m_slice->m_refFrameList[0][0]->m_encData->m_pir.pirEndCol * g_maxCUSize - 3;<br>
> >> + maxSafeMv = (safeX - tempPred->cu.m_cuPelX) * 4;<br>
> >> + }<br>
> >> for (uint32_t i = 0; i < numMergeCand; ++i)<br>
> >> {<br>
> >> if (m_bFrameParallel &&<br>
> >> (candMvField[i][0].mv.y >= (m_param->searchRange + 1) * 4 ||<br>
> >> candMvField[i][1].mv.y >= (m_param->searchRange + 1) * 4))<br>
> >> continue;<br>
> >> + if (m_param->bIntraRefresh && m_slice->m_sliceType == P_SLICE &&<br>
> >> + tempPred->cu.m_cuPelX / g_maxCUSize <<br>
> >> m_frame->m_encData->m_pir.pirEndCol &&<br>
> >> + candMvField[i][0].mv.x > maxSafeMv)<br>
> >> + // skip merge candidates which reference beyond safe<br>
> >> reference area<br>
> >> + continue;<br>
> >><br>
> >> tempPred->cu.m_mvpIdx[0][0] = (uint8_t)i; // merge candidate ID<br>
> >> is stored in L0 MVP idx<br>
> >> X265_CHECK(m_slice->m_sliceType == B_SLICE || !(candDir[i] &<br>
> >> 0x10), " invalid merge for P slice\n");<br>
> >> @@ -1570,7 +1584,12 @@<br>
> >> first = *m_reuseBestMergeCand;<br>
> >> last = first + 1;<br>
> >> }<br>
> >> -<br>
> >> + int safeX, maxSafeMv;<br>
> >> + if (m_param->bIntraRefresh && m_slice->m_sliceType == P_SLICE)<br>
> >> + {<br>
> >> + safeX =<br>
> >> m_slice->m_refFrameList[0][0]->m_encData->m_pir.pirEndCol * g_maxCUSize - 3;<br>
> >> + maxSafeMv = (safeX - tempPred->cu.m_cuPelX) * 4;<br>
> >> + }<br>
> >> for (uint32_t i = first; i < last; i++)<br>
> >> {<br>
> >> if (m_bFrameParallel &&<br>
> >> @@ -1593,7 +1612,11 @@<br>
> >> continue;<br>
> >> triedBZero = true;<br>
> >> }<br>
> >> -<br>
> >> + if (m_param->bIntraRefresh && m_slice->m_sliceType == P_SLICE &&<br>
> >> + tempPred->cu.m_cuPelX / g_maxCUSize <<br>
> >> m_frame->m_encData->m_pir.pirEndCol &&<br>
> >> + candMvField[i][0].mv.x > maxSafeMv)<br>
> >> + // skip merge candidates which reference beyond safe<br>
> >> reference area<br>
> >> + continue;<br>
> >> tempPred->cu.m_mvpIdx[0][0] = (uint8_t)i; /* merge candidate<br>
> >> ID is stored in L0 MVP idx */<br>
> >> tempPred->cu.m_interDir[0] = candDir[i];<br>
> >> tempPred->cu.m_mv[0][0] = candMvField[i][0].mv;<br>
> >> diff -r 98c0dcd5a10b -r 97b62cb57bfa source/encoder/api.cpp<br>
> >> --- a/source/encoder/api.cpp Wed Sep 09 14:52:35 2015 +0530<br>
> >> +++ b/source/encoder/api.cpp Wed Sep 16 14:26:59 2015 +0530<br>
> >> @@ -245,6 +245,16 @@<br>
> >> }<br>
> >> }<br>
> >><br>
> >> +int x265_encoder_intra_refresh(x265_encoder *enc)<br>
> >> +{<br>
> >> + if (!enc)<br>
> >> + return -1;<br>
> >> +<br>
> >> + Encoder *encoder = static_cast<Encoder*>(enc);<br>
> >> + encoder->m_param->bQueuedIntraRefresh = 1;<br>
> >> + return 0;<br>
> >> +}<br>
> >> +<br>
> >> void x265_cleanup(void)<br>
> >> {<br>
> >> if (!g_ctuSizeConfigured)<br>
> >> @@ -317,6 +327,7 @@<br>
> >> &x265_cleanup,<br>
> >><br>
> >> sizeof(x265_frame_stats),<br>
> >> + &x265_encoder_intra_refresh,<br>
> >> };<br>
> >><br>
> >> typedef const x265_api* (*api_get_func)(int bitDepth);<br>
> >> diff -r 98c0dcd5a10b -r 97b62cb57bfa source/encoder/encoder.cpp<br>
> >> --- a/source/encoder/encoder.cpp Wed Sep 09 14:52:35 2015 +0530<br>
> >> +++ b/source/encoder/encoder.cpp Wed Sep 16 14:26:59 2015 +0530<br>
> >> @@ -437,6 +437,46 @@<br>
> >> }<br>
> >> }<br>
> >><br>
> >> +void Encoder::calcRefreshInterval(Frame* frameEnc)<br>
> >> +{<br>
> >> + Slice* slice = frameEnc->m_encData->m_slice;<br>
> >> + uint32_t numBlocksInRow = slice->m_sps->numCuInWidth;<br>
> >> + FrameData::PeriodicIR* pir = &frameEnc->m_encData->m_pir;<br>
> >> + if (slice->m_sliceType == I_SLICE)<br>
> >> + {<br>
> >> + pir->framesSinceLastPir = 0;<br>
> >> + m_param->bQueuedIntraRefresh = 0;<br>
> >> + /* PIR is currently only supported with ref == 1, so any intra<br>
> >> frame effectively refreshes<br>
> >> + * the whole frame and counts as an intra refresh. */<br>
> >> + pir->position = numBlocksInRow;<br>
> >> + }<br>
> >> + else if (slice->m_sliceType == P_SLICE)<br>
> >> + {<br>
> >> + Frame* ref = frameEnc->m_encData->m_slice->m_refFrameList[0][0];<br>
> >> + int pocdiff = frameEnc->m_poc - ref->m_poc;<br>
> >> + float increment = X265_MAX(((float)numBlocksInRow - 1) /<br>
> >> m_param->keyframeMax, 1);<br>
> >> + pir->position = ref->m_encData->m_pir.position;<br>
> >> + pir->framesSinceLastPir =<br>
> >> ref->m_encData->m_pir.framesSinceLastPir + pocdiff;<br>
> >> + if (pir->framesSinceLastPir >= m_param->keyframeMax ||<br>
> >> + (m_param->bQueuedIntraRefresh && pir->position + 0.5 >=<br>
> >> numBlocksInRow))<br>
> >> + {<br>
> >> + pir->position = 0;<br>
> >> + pir->framesSinceLastPir = 0;<br>
> >> + m_param->bQueuedIntraRefresh = 0;<br>
> >> + frameEnc->m_lowres.bKeyframe = 1;<br>
> >> + }<br>
> >> + pir->pirStartCol = (uint32_t)(pir->position + 0.5);<br>
> >> + pir->position += increment * pocdiff;<br>
> >> + pir->pirEndCol = (uint32_t)(pir->position + 0.5);<br>
> >> + /* If our intra refresh has reached the right side of the frame,<br>
> >> we're done. */<br>
> >> + if (pir->pirEndCol >= numBlocksInRow)<br>
> >> + {<br>
> >> + pir->position = numBlocksInRow;<br>
> >> + pir->pirEndCol = numBlocksInRow;<br>
> >> + }<br>
> >> + }<br>
> >> +}<br>
> >> +<br>
> >> /**<br>
> >> * Feed one new input frame into the encoder, get one frame out. If<br>
> >> pic_in is<br>
> >> * NULL, a flush condition is implied and pic_in must be NULL for all<br>
> >> subsequent<br>
> >> @@ -768,6 +808,8 @@<br>
> >><br>
> >> if (m_param->rc.rateControlMode != X265_RC_CQP)<br>
> >> m_lookahead->getEstimatedPictureCost(frameEnc);<br>
> >> + if (m_param->bIntraRefresh)<br>
> >> + calcRefreshInterval(frameEnc);<br>
> >><br>
> >> /* Allow FrameEncoder::compressFrame() to start in the frame<br>
> >> encoder thread */<br>
> >> if (!curEncoder->startCompressFrame(frameEnc))<br>
> >> diff -r 98c0dcd5a10b -r 97b62cb57bfa source/encoder/encoder.h<br>
> >> --- a/source/encoder/encoder.h Wed Sep 09 14:52:35 2015 +0530<br>
> >> +++ b/source/encoder/encoder.h Wed Sep 16 14:26:59 2015 +0530<br>
> >> @@ -168,6 +168,8 @@<br>
> >><br>
> >> void finishFrameStats(Frame* pic, FrameEncoder *curEncoder, uint64_t<br>
> >> bits, x265_frame_stats* frameStats);<br>
> >><br>
> >> + void calcRefreshInterval(Frame* frameEnc);<br>
> >> +<br>
> >> protected:<br>
> >><br>
> >> void initVPS(VPS *vps);<br>
> >> diff -r 98c0dcd5a10b -r 97b62cb57bfa source/encoder/search.cpp<br>
> >> --- a/source/encoder/search.cpp Wed Sep 09 14:52:35 2015 +0530<br>
> >> +++ b/source/encoder/search.cpp Wed Sep 16 14:26:59 2015 +0530<br>
> >> @@ -2460,6 +2460,17 @@<br>
> >> cu.clipMv(mvmin);<br>
> >> cu.clipMv(mvmax);<br>
> >><br>
> >> + if (cu.m_encData->m_param->bIntraRefresh && m_slice->m_sliceType ==<br>
> >> P_SLICE &&<br>
> >> + cu.m_cuPelX / g_maxCUSize <<br>
> >> m_frame->m_encData->m_pir.pirStartCol &&<br>
> >> + m_slice->m_refFrameList[0][0]->m_encData->m_pir.pirEndCol <<br>
> >> m_slice->m_sps->numCuInWidth)<br>
> >><br>
> ><br>
> > Why check for m_slice->m_refFrameList[0][0]->m_encData->m_pir.pirEndCol <<br>
> > m_slice->m_sps->numCuInWidth here? This should be taken care of in<br>
> > calcRefreshInterval.<br>
> ><br>
><br>
> In calcRefreshInterval, pirEndCol is set with value which indicates the<br>
> maximum edge of the safe region for reference in intra refresh. Once we<br>
> reach the right side of the frame, we signal it in<br>
> pirEndCol by setting it with numCuInWidth. Indicating that we can no longer<br>
> use this for reference during intra refresh. So in places where we set max<br>
> and min mv range, it is necessary to check this condition. We anyway need<br>
> another flag if we want this check in calcRefreshInterval.<br>
<br>
</div></div>More than that, if the reference frame was not part of a refresh sweep,<br>
we do not want to stop motion searches at the right edge. There is a<br>
difference between limiting the ME range to the right picture edge and<br>
having no limit at all (aka, the right extended border). Searches need<br>
to be able to cross the picture edge<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Steve Borho<br>
</font></span><div class="HOEnZb"><div class="h5">_______________________________________________<br>
x265-devel mailing list<br>
<a href="mailto:x265-devel@videolan.org">x265-devel@videolan.org</a><br>
<a href="https://mailman.videolan.org/listinfo/x265-devel" rel="noreferrer" target="_blank">https://mailman.videolan.org/listinfo/x265-devel</a><br>
</div></div></blockquote></div><br></div>