<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Sep 14, 2015 at 9:16 PM, Steve Borho <span dir="ltr"><<a href="mailto:steve@borho.org" target="_blank">steve@borho.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On 09/14, <a href="mailto:santhoshini@multicorewareinc.com">santhoshini@multicorewareinc.com</a> wrote:<br>
> # HG changeset patch<br>
> # User Santhoshini Sekar<<a href="mailto:santhoshini@multicorewareinc.com">santhoshini@multicorewareinc.com</a>><br>
> # Date 1442213046 -19800<br>
> # Mon Sep 14 12:14:06 2015 +0530<br>
> # Node ID 0eb755da6cab80020bf26410438d83337f9aed9a<br>
> # Parent f6892dcc7f4e74af58d7f39e085aed44afba919c<br>
> Implementation for Intra-refresh<br>
><br>
> diff -r f6892dcc7f4e -r 0eb755da6cab source/encoder/analysis.cpp<br>
> --- a/source/encoder/analysis.cpp Wed Sep 09 14:52:35 2015 +0530<br>
> +++ b/source/encoder/analysis.cpp Mon Sep 14 12:14:06 2015 +0530<br>
> @@ -170,7 +170,12 @@<br>
> }<br>
> else<br>
> {<br>
> - if (!m_param->rdLevel)<br>
> + int bForceIntra = m_param->bIntraRefresh && m_slice->m_sliceType == P_SLICE &&<br>
> + ctu.m_cuPelX / g_maxCUSize >= frame.m_encData->m_pir.pirStartPelX && ctu.m_cuPelX / g_maxCUSize < frame.m_encData->m_pir.pirEndPelX;<br>
> +<br>
> + if (bForceIntra)<br>
> + compressIntraCU(ctu, cuGeom, zOrder, qp);<br>
> + else if (!m_param->rdLevel)<br>
> {<br>
> /* In RD Level 0/1, copy source pixels into the reconstructed block so<br>
> * they are available for intra predictions */<br>
> @@ -1464,7 +1469,13 @@<br>
> (candMvField[i][0].mv.y >= (m_param->searchRange + 1) * 4 ||<br>
> candMvField[i][1].mv.y >= (m_param->searchRange + 1) * 4))<br>
> continue;<br>
> -<br>
> + if (m_param->bIntraRefresh && m_slice->m_sliceType == P_SLICE)<br>
> + {<br>
> + int maxX = (m_slice->m_refFrameList[0][0]->m_encData->m_pir.pirEndPelX * g_maxCUSize - 3) * 4;<br>
<br>
</div></div>I'm not following this math, is m_pir.pirEndPelX not in units of PELs<br>
(pixels)? The multiplication by g_maxCUSize would indicate it's in<br>
units of CTUs. And what is 3? is that the half-filter width? if so it<br>
needs to be adjusted for HEVC's filter-width (and probably use the<br>
existing macro)<br>
<div><div class="h5"><br>
> + int maxMv = maxX - 4 * g_maxCUSize * candMvField[i][0].mv.x;<br>
> + if (maxMv > 0 && tempPred->cu.m_cuPelX / g_maxCUSize < m_frame->m_encData->m_pir.pirEndPelX)<br>
> + continue;<br>
> + }<br>
> tempPred->cu.m_mvpIdx[0][0] = (uint8_t)i; // merge candidate ID is stored in L0 MVP idx<br>
> X265_CHECK(m_slice->m_sliceType == B_SLICE || !(candDir[i] & 0x10), " invalid merge for P slice\n");<br>
> tempPred->cu.m_interDir[0] = candDir[i];<br>
> @@ -1592,7 +1603,13 @@<br>
> continue;<br>
> triedBZero = true;<br>
> }<br>
> -<br>
> + if (m_param->bIntraRefresh && m_slice->m_sliceType == P_SLICE)<br>
> + {<br>
> + int maxX = (m_slice->m_refFrameList[0][0]->m_encData->m_pir.pirEndPelX * g_maxCUSize - 3) * 4;<br>
> + int maxMv = maxX - 4 * g_maxCUSize * candMvField[i][0].mv.x;<br>
> + if (maxMv > 0 && tempPred->cu.m_cuPelX / g_maxCUSize < m_frame->m_encData->m_pir.pirEndPelX)<br>
> + continue;<br>
> + }<br>
> tempPred->cu.m_mvpIdx[0][0] = (uint8_t)i; /* merge candidate ID is stored in L0 MVP idx */<br>
> tempPred->cu.m_interDir[0] = candDir[i];<br>
> tempPred->cu.m_mv[0][0] = candMvField[i][0].mv;<br>
> diff -r f6892dcc7f4e -r 0eb755da6cab source/encoder/api.cpp<br>
> --- a/source/encoder/api.cpp Wed Sep 09 14:52:35 2015 +0530<br>
> +++ b/source/encoder/api.cpp Mon Sep 14 12:14:06 2015 +0530<br>
> @@ -245,6 +245,16 @@<br>
> }<br>
> }<br>
><br>
> +int x265_encoder_intra_refresh(x265_encoder *enc)<br>
> +{<br>
> + if (!enc)<br>
> + return -1;<br>
> +<br>
> + Encoder *encoder = static_cast<Encoder*>(enc);<br>
> + encoder->m_param->bQueuedIntraRefresh = 1;<br>
> + return 0;<br>
> +}<br>
> +<br>
> void x265_cleanup(void)<br>
> {<br>
> if (!g_ctuSizeConfigured)<br>
> @@ -315,6 +325,7 @@<br>
> &x265_encoder_log,<br>
> &x265_encoder_close,<br>
> &x265_cleanup,<br>
> + &x265_encoder_intra_refresh,<br>
><br>
> sizeof(x265_frame_stats),<br>
> };<br>
> diff -r f6892dcc7f4e -r 0eb755da6cab source/encoder/encoder.cpp<br>
> --- a/source/encoder/encoder.cpp Wed Sep 09 14:52:35 2015 +0530<br>
> +++ b/source/encoder/encoder.cpp Mon Sep 14 12:14:06 2015 +0530<br>
> @@ -768,6 +768,43 @@<br>
> if (m_param->rc.rateControlMode != X265_RC_CQP)<br>
> m_lookahead->getEstimatedPictureCost(frameEnc);<br>
><br>
> + if (m_param->bIntraRefresh)<br>
> + {<br>
> + Slice* slice = frameEnc->m_encData->m_slice;<br>
<br>
</div></div>this code would be more readable if you made a reference to<br>
frameEnc->m_encData->m_pir<br>
<span class=""><br>
> + if (slice->m_sliceType == I_SLICE)<br>
> + {<br>
> + frameEnc->m_encData->m_pir.framesSinceLastPir = 0;<br>
> + m_param->bQueuedIntraRefresh = 0;<br>
> + /* PIR is currently only supported with ref == 1, so any intra frame effectively refreshes<br>
> + * the whole frame and counts as an intra refresh. */<br>
> + frameEnc->m_encData->m_pir.position = frameEnc->m_lowres.maxBlocksInRow;<br>
> + }<br>
> + else if (slice->m_sliceType == P_SLICE)<br>
> + {<br>
> + Frame* ref = frameEnc->m_encData->m_slice->m_refFrameList[0][0];<br>
> + int pocdiff = (frameEnc->m_poc - ref->m_poc);<br>
<br>
</span>parens are unnecessary<br>
<div><div class="h5"><br>
> + float increment = X265_MAX(((float)frameEnc->m_lowres.maxBlocksInRow - 1) / m_param->keyframeMax, 1);<br>
> + frameEnc->m_encData->m_pir.position = ref->m_encData->m_pir.position;<br>
> + frameEnc->m_encData->m_pir.framesSinceLastPir = ref->m_encData->m_pir.framesSinceLastPir + pocdiff;<br>
> + if (frameEnc->m_encData->m_pir.framesSinceLastPir >= m_param->keyframeMax ||<br>
> + (m_param->bQueuedIntraRefresh && frameEnc->m_encData->m_pir.position + 0.5 >= frameEnc->m_lowres.maxBlocksInRow))<br>
> + {<br>
> + frameEnc->m_encData->m_pir.position = 0;<br>
> + frameEnc->m_encData->m_pir.framesSinceLastPir = 0;<br>
> + m_param->bQueuedIntraRefresh = 0;<br>
> + frameEnc->m_lowres.bKeyframe = 1;<br>
> + }<br>
> + frameEnc->m_encData->m_pir.pirStartPelX = (uint32_t)(frameEnc->m_encData->m_pir.position + 0.5);<br>
> + frameEnc->m_encData->m_pir.position += increment * pocdiff;<br>
> + frameEnc->m_encData->m_pir.pirEndPelX = (uint32_t)(frameEnc->m_encData->m_pir.position + 0.5);<br>
> + /* If our intra refresh has reached the right side of the frame, we're done. */<br>
> + if (frameEnc->m_encData->m_pir.pirEndPelX >= frameEnc->m_lowres.maxBlocksInRow - 1)<br>
> + {<br>
> + frameEnc->m_encData->m_pir.position = frameEnc->m_lowres.maxBlocksInRow;<br>
> + frameEnc->m_encData->m_pir.pirEndPelX = frameEnc->m_lowres.maxBlocksInRow - 1;<br>
> + }<br>
> + }<br>
> + }<br>
> /* Allow FrameEncoder::compressFrame() to start in the frame encoder thread */<br>
> if (!curEncoder->startCompressFrame(frameEnc))<br>
> m_aborted = true;<br>
> diff -r f6892dcc7f4e -r 0eb755da6cab source/encoder/search.cpp<br>
> --- a/source/encoder/search.cpp Wed Sep 09 14:52:35 2015 +0530<br>
> +++ b/source/encoder/search.cpp Mon Sep 14 12:14:06 2015 +0530<br>
> @@ -1923,6 +1923,26 @@<br>
> return costs[0] <= costs[1] ? 0 : 1;<br>
> }<br>
><br>
> +void Search::clipMvPIR(const CUData& cu, MV& outMV) const<br>
> +{<br>
> + const uint32_t mvshift = 2;<br>
> + uint32_t offset = 8;<br>
> +<br>
> + int16_t xmax = (int16_t)((m_slice->m_sps->picWidthInLumaSamples + offset - cu.m_cuPelX - 1) << mvshift);<br>
> + int16_t xmin = -(int16_t)((g_maxCUSize + offset + cu.m_cuPelX - 1) << mvshift);<br>
> +<br>
> + int16_t ymax = (int16_t)((m_slice->m_sps->picHeightInLumaSamples + offset - cu.m_cuPelY - 1) << mvshift);<br>
> + int16_t ymin = -(int16_t)((g_maxCUSize + offset + cu.m_cuPelY - 1) << mvshift);<br>
> +<br>
> + int maxX = (m_slice->m_refFrameList[0][0]->m_encData->m_pir.pirEndPelX * g_maxCUSize - 3) << mvshift;<br>
> + int maxMv = maxX - 4* g_maxCUSize * outMV.x;<br>
> + if (maxMv > 0 && cu.m_cuPelX / g_maxCUSize < m_frame->m_encData->m_pir.pirStartPelX)<br>
> + xmax = X265_MIN(xmax, (int16_t)maxX);<br>
> +<br>
> + outMV.x = X265_MIN(xmax, X265_MAX(xmin, outMV.x));<br>
> + outMV.y = X265_MIN(ymax, X265_MAX(ymin, outMV.y));<br>
> +}<br>
> +<br>
> void Search::PME::processTasks(int workerThreadId)<br>
> {<br>
> #if DETAILED_CU_STATS<br>
> @@ -2088,6 +2108,11 @@<br>
> mvc[numMvc++] = lmv;<br>
><br>
> setSearchRange(cu, mvp, m_param->searchRange, mvmin, mvmax);<br>
> + if (m_param->bIntraRefresh && m_slice->m_sliceType == P_SLICE)<br>
> + {<br>
> + clipMvPIR(cu, mvmin);<br>
> + clipMvPIR(cu, mvmax);<br>
> + }<br>
<br>
</div></div>there is a lot of code duplication between setSearchRange() and<br>
clipMvPIR(). Should the two be combined?<br>
<br>
is setSearchRange() called in more places?<br></blockquote><div>Yes setSearchRange() is called in many places wherein clipMvPIR() is not needed. Initially I had the functionality of clipMvPIR() in clipMV(). But, later <br>added this new function as clipMV is also called in multiple places.</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div class="h5"><br>
> int satdCost = m_me.motionEstimate(&slice->m_mref[list][ref], mvmin, mvmax, mvp, numMvc, mvc, m_param->searchRange, outmv);<br>
><br>
> /* Get total cost of partition, but only include MV bit cost once */<br>
> diff -r f6892dcc7f4e -r 0eb755da6cab source/encoder/search.h<br>
> --- a/source/encoder/search.h Wed Sep 09 14:52:35 2015 +0530<br>
> +++ b/source/encoder/search.h Mon Sep 14 12:14:06 2015 +0530<br>
> @@ -449,6 +449,7 @@<br>
><br>
> /* inter/ME helper functions */<br>
> int selectMVP(const CUData& cu, const PredictionUnit& pu, const MV amvp[AMVP_NUM_CANDS], int list, int ref);<br>
> + void clipMvPIR(const CUData& cu, MV& outMV) const; // used only for Periodic Intra refresh<br>
> const MV& checkBestMVP(const MV amvpCand[2], const MV& mv, int& mvpIdx, uint32_t& outBits, uint32_t& outCost) const;<br>
> void setSearchRange(const CUData& cu, const MV& mvp, int merange, MV& mvmin, MV& mvmax) const;<br>
> uint32_t mergeEstimation(CUData& cu, const CUGeom& cuGeom, const PredictionUnit& pu, int puIdx, MergeData& m);<br>
> diff -r f6892dcc7f4e -r 0eb755da6cab source/encoder/slicetype.cpp<br>
> --- a/source/encoder/slicetype.cpp Wed Sep 09 14:52:35 2015 +0530<br>
> +++ b/source/encoder/slicetype.cpp Mon Sep 14 12:14:06 2015 +0530<br>
> @@ -774,6 +774,7 @@<br>
> for (uint32_t cnt = 0; cnt < scale && lowresRow < heightInLowresCu; lowresRow++, cnt++)<br>
> {<br>
> sum = 0; intraSum = 0;<br>
> + int diff = 0;<br>
> lowresCuIdx = lowresRow * widthInLowresCu;<br>
> for (lowresCol = 0; lowresCol < widthInLowresCu; lowresCol++, lowresCuIdx++)<br>
> {<br>
> @@ -781,14 +782,18 @@<br>
> if (qp_offset)<br>
> {<br>
> lowresCuCost = (uint16_t)((lowresCuCost * x265_exp2fix8(qp_offset[lowresCuIdx]) + 128) >> 8);<br>
> - int32_t intraCuCost = curFrame->m_lowres.intraCost[lowresCuIdx];<br>
> + int32_t intraCuCost = curFrame->m_lowres.intraCost[lowresCuIdx];<br>
> curFrame->m_lowres.intraCost[lowresCuIdx] = (intraCuCost * x265_exp2fix8(qp_offset[lowresCuIdx]) + 128) >> 8;<br>
> }<br>
> + if (m_param->bIntraRefresh && slice->m_sliceType == X265_TYPE_P)<br>
> + for (uint32_t x = curFrame->m_encData->m_pir.pirStartPelX; x <= curFrame->m_encData->m_pir.pirEndPelX; x++)<br>
> + diff += curFrame->m_lowres.intraCost[lowresCuIdx] - lowresCuCost;<br>
> curFrame->m_lowres.lowresCostForRc[lowresCuIdx] = lowresCuCost;<br>
> sum += lowresCuCost;<br>
> intraSum += curFrame->m_lowres.intraCost[lowresCuIdx];<br>
> }<br>
> curFrame->m_encData->m_rowStat[row].satdForVbv += sum;<br>
> + curFrame->m_encData->m_rowStat[row].satdForVbv += diff;<br>
> curFrame->m_encData->m_rowStat[row].intraSatdForVbv += intraSum;<br>
> }<br>
> }<br>
> @@ -900,8 +905,7 @@<br>
> x265_log(m_param, X265_LOG_WARNING, "B-ref at frame %d incompatible with B-pyramid and %d reference frames\n",<br>
> frm.sliceType, m_param->maxNumReferences);<br>
> }<br>
> -<br>
> - if (/* (!param->intraRefresh || frm.frameNum == 0) && */ frm.frameNum - m_lastKeyframe >= m_param->keyframeMax)<br>
> + if ((!m_param->bIntraRefresh || frm.frameNum == 0) && frm.frameNum - m_lastKeyframe >= m_param->keyframeMax)<br>
> {<br>
> if (frm.sliceType == X265_TYPE_AUTO || frm.sliceType == X265_TYPE_I)<br>
> frm.sliceType = m_param->bOpenGOP && m_lastKeyframe >= 0 ? X265_TYPE_I : X265_TYPE_IDR;<br>
> @@ -1184,7 +1188,7 @@<br>
> frames[framecnt + 1] = NULL;<br>
><br>
> keyintLimit = m_param->keyframeMax - frames[0]->frameNum + m_lastKeyframe - 1;<br>
> - origNumFrames = numFrames = X265_MIN(framecnt, keyintLimit);<br>
> + origNumFrames = numFrames = m_param->bIntraRefresh ? framecnt : X265_MIN(framecnt, keyintLimit);<br>
><br>
> if (bIsVbvLookahead)<br>
> numFrames = framecnt;<br>
> @@ -1384,12 +1388,12 @@<br>
> if (m_param->rc.cuTree)<br>
> cuTree(frames, X265_MIN(numFrames, m_param->keyframeMax), bKeyframe);<br>
><br>
> - // if (!param->bIntraRefresh)<br>
> - for (int j = keyintLimit + 1; j <= numFrames; j += m_param->keyframeMax)<br>
> - {<br>
> - frames[j]->sliceType = X265_TYPE_I;<br>
> - resetStart = X265_MIN(resetStart, j + 1);<br>
> - }<br>
> + if (!m_param->bIntraRefresh)<br>
> + for (int j = keyintLimit + 1; j <= numFrames; j += m_param->keyframeMax)<br>
> + {<br>
> + frames[j]->sliceType = X265_TYPE_I;<br>
> + resetStart = X265_MIN(resetStart, j + 1);<br>
> + }<br>
><br>
> if (bIsVbvLookahead)<br>
> vbvLookahead(frames, numFrames, bKeyframe);<br>
> @@ -1504,7 +1508,7 @@<br>
> {<br>
> if (m_param->keyframeMin == m_param->keyframeMax)<br>
> threshMin = threshMax;<br>
> - if (gopSize <= m_param->keyframeMin / 4)<br>
> + if (gopSize <= m_param->keyframeMin / 4 || m_param->bIntraRefresh)<br>
> bias = threshMin / 4;<br>
> else if (gopSize <= m_param->keyframeMin)<br>
> bias = threshMin * gopSize / m_param->keyframeMin;<br>
> diff -r f6892dcc7f4e -r 0eb755da6cab source/x265.h<br>
> --- a/source/x265.h Wed Sep 09 14:52:35 2015 +0530<br>
> +++ b/source/x265.h Mon Sep 14 12:14:06 2015 +0530<br>
> @@ -707,6 +707,8 @@<br>
> * big keyframe, the keyframe is "spread" over many frames. */<br>
> int bIntraRefresh;<br>
><br>
> + int bQueuedIntraRefresh;<br>
> +<br>
> /*== Coding Unit (CU) definitions ==*/<br>
><br>
> /* Maximum CU width and height in pixels. The size must be 64, 32, or 16.<br>
> @@ -1378,6 +1380,22 @@<br>
> * close an encoder handler */<br>
> void x265_encoder_close(x265_encoder *);<br>
><br>
> +/* x265_encoder_intra_refresh:<br>
> + * If an intra refresh is not in progress, begin one with the next P-frame.<br>
> + * If an intra refresh is in progress, begin one as soon as the current one finishes.<br>
> + * Requires bIntraRefresh to be set.<br>
> + *<br>
> + * Useful for interactive streaming where the client can tell the server that packet loss has<br>
> + * occurred. In this case, keyint can be set to an extremely high value so that intra refreshes<br>
> + * occur only when calling x265_encoder_intra_refresh.<br>
> + *<br>
> + * In multi-pass encoding, if x265_encoder_intra_refresh is called differently in each pass,<br>
> + * behavior is undefined.<br>
> + *<br>
> + * Should not be called during an x265_encoder_encode. */<br>
> +<br>
> +int x265_encoder_intra_refresh(x265_encoder *);<br>
> +<br>
> /* x265_cleanup:<br>
> * release library static allocations, reset configured CTU size */<br>
> void x265_cleanup(void);<br>
> @@ -1423,6 +1441,7 @@<br>
> void (*encoder_log)(x265_encoder*, int, char**);<br>
> void (*encoder_close)(x265_encoder*);<br>
> void (*cleanup)(void);<br>
> + int (*encoder_intra_refresh)(x265_encoder*);<br>
><br>
> int sizeof_frame_stats; /* sizeof(x265_frame_stats) */<br>
> /* add new pointers to the end, or increment X265_MAJOR_VERSION */<br>
<br>
</div></div>x265_encoder_intra_refresh needs to be added to <a href="http://x265.def.in" rel="noreferrer" target="_blank">x265.def.in</a><br>
<div class="HOEnZb"><div class="h5"><br>
--<br>
Steve Borho<br>
_______________________________________________<br>
x265-devel mailing list<br>
<a href="mailto:x265-devel@videolan.org">x265-devel@videolan.org</a><br>
<a href="https://mailman.videolan.org/listinfo/x265-devel" rel="noreferrer" target="_blank">https://mailman.videolan.org/listinfo/x265-devel</a><br>
</div></div></blockquote></div><br></div></div>