<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jul 22, 2015 at 11:08 PM, Steve Borho <span dir="ltr"><<a href="mailto:steve@borho.org" target="_blank">steve@borho.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On 07/22, Aarthi Priya Thirumalai wrote:<br>
> Currently during scene change transitions, esp slow gradual<br>
> fadeins/fadeouts every frame changes vastly in luma intensity and hence<br>
> passes as a scenecut but these are treated<br>
> as a series of flashes in scenecut() . So, none of the frames gets marked<br>
> as scencut during such transitions.<br>
> It would be better if we can signal the first frame of such transition<br>
> flashes as sceneChange for RateControl. so that the rate control can set a<br>
> minimum qp for such frames.<br>
> This would help the bitrate spikes at fade ins where the qp can be really<br>
> low.<br>
<br>
</span>weightp in the lookahead doesn't help for fades? or is this a problem in<br>
presets with weightp disabled?<br></blockquote><div><br></div><div>weightp doesn't help for fades. happens even when weightp is enabled. </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span class=""><br>
> The patch mainly fixes the place in scenecut() where the flash detection<br>
> happens and the first frame in a series of flash is signaled as<br>
> sceneChange.<br>
> Changing bias or modifying scenecutInterval doesn't help here as those<br>
> frames are already ,marked as scenecuts from those function.<br>
> Only scencut() flash detection part reverts the flashes as non-scenecuts<br>
> which we are changing.<br>
><br>
> the slicetype decision is not changed as I am signalling these scene<br>
> changes with a different flag that Ratecontrol would be using.<br>
<br>
</span>this problem of fades being detected as flashes is probably more<br>
prevelant when the keyframe distance approaches the limit, the bias<br>
ensures this. it seems like both sides of it need to be addressed.<br></blockquote><div><br></div><div>actually even when the keyframe interval is not closer to the transition, they get detected</div><div>as flashes only. for each frame in the fadein, the satdcosts are strong wnough to overcome the bias and get detected as scenecut. I experimented with constant bias throughout (remove keyframe effect from bias calculation) yet the fades are getting detected as a series of flashes. </div><div>Hence I am trying to signal the first frame in a series of flashes for rate control rather than the last frame as currently signalled by scenecut(). since I am using a different flag,current slice type decisions are not affected. </div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span class=""><br>
> On Wed, Jul 22, 2015 at 10:10 PM, Deepthi Nandakumar <<br>
</span><span class="">> <a href="mailto:deepthi@multicorewareinc.com">deepthi@multicorewareinc.com</a>> wrote:<br>
><br>
> > The intention was to separate slicetype decision from scene change<br>
> > detection, where scene change detection would also account for transitions<br>
> > (like fade-ins, fade-outs etc), in addition to the abrupt scene change.<br>
> ><br>
> > It is not clear to me how this is being done without modifying<br>
> > scenecutInternal.<br>
> ><br>
</span>> > On Wed, Jul 22, 2015 at 9:59 PM, Steve Borho <<a href="mailto:steve@borho.org">steve@borho.org</a>> wrote:<br>
<div><div class="h5">> ><br>
> >> On 07/22, <a href="mailto:aarthi@multicorewareinc.com">aarthi@multicorewareinc.com</a> wrote:<br>
> >> > # HG changeset patch<br>
> >> > # User Aarthi Thirumalai<br>
> >> > # Date 1437505166 -19800<br>
> >> > # Wed Jul 22 00:29:26 2015 +0530<br>
> >> > # Node ID f134920ac855af45f052ccd6d3c239d39940a100<br>
> >> > # Parent 46152345eb6ff261fd90272f7a0712300d6324c0<br>
> >> > slicetype: Modify Scenecut algorithm to detect scene transition points<br>
> >> > to improve Rate Control.<br>
> >> ><br>
> >> > identify scene trasitions, fade-ins, fadeouts, sceneCuts and signal the<br>
> >> flag bSceneChange<br>
> >> > in Lowres structure. This flag will be used by RateControl to adjust<br>
> >> the wps during scene cuts.<br>
> >> > No changes in sliceType decisions or I frame placement.<br>
> >><br>
> >> I could nitpick a few coding style issues, but my main question is<br>
> >> whether you tried disabling the keyframe distance bias during flash<br>
> >> detection and arrived at this approach, or whether this is an orthogonal<br>
> >> change?<br>
> >><br>
> >> > diff -r 46152345eb6f -r f134920ac855 source/common/lowres.cpp<br>
> >> > --- a/source/common/lowres.cpp Mon Jul 20 17:18:54 2015 -0700<br>
> >> > +++ b/source/common/lowres.cpp Wed Jul 22 00:29:26 2015 +0530<br>
> >> > @@ -127,6 +127,7 @@<br>
> >> > {<br>
> >> > bLastMiniGopBFrame = false;<br>
> >> > bScenecut = true; // could be a scene-cut, until ruled out by<br>
> >> flash detection<br>
> >> > + bSceneChange = true;<br>
> >> > bKeyframe = false; // Not a keyframe unless identified by lookahead<br>
> >> > frameNum = poc;<br>
> >> > leadingBframes = 0;<br>
> >> > diff -r 46152345eb6f -r f134920ac855 source/common/lowres.h<br>
> >> > --- a/source/common/lowres.h Mon Jul 20 17:18:54 2015 -0700<br>
> >> > +++ b/source/common/lowres.h Wed Jul 22 00:29:26 2015 +0530<br>
> >> > @@ -115,6 +115,7 @@<br>
> >> > int leadingBframes; // number of leading B frames for P or I<br>
> >> ><br>
> >> > bool bScenecut; // Set to false if the frame cannot<br>
> >> possibly be part of a real scenecut.<br>
> >> > + bool bSceneChange;<br>
> >> > bool bKeyframe;<br>
> >> > bool bLastMiniGopBFrame;<br>
> >> ><br>
> >> > diff -r 46152345eb6f -r f134920ac855 source/encoder/slicetype.cpp<br>
> >> > --- a/source/encoder/slicetype.cpp Mon Jul 20 17:18:54 2015 -0700<br>
> >> > +++ b/source/encoder/slicetype.cpp Wed Jul 22 00:29:26 2015 +0530<br>
> >> > @@ -462,6 +462,7 @@<br>
> >> > m_pool = pool;<br>
> >> ><br>
> >> > m_lastNonB = NULL;<br>
> >> > + m_isSceneTransition = false;<br>
> >> > m_scratch = NULL;<br>
> >> > m_tld = NULL;<br>
> >> > m_filled = false;<br>
> >> > @@ -1375,6 +1376,8 @@<br>
> >> > bool Lookahead::scenecut(Lowres **frames, int p0, int p1, bool<br>
> >> bRealScenecut, int numFrames, int maxSearch)<br>
> >> > {<br>
> >> > /* Only do analysis during a normal scenecut check. */<br>
> >> > + int64_t avgSatdCost = 0;<br>
> >> > + int cnt = 0;<br>
> >> > if (bRealScenecut && m_param->bframes)<br>
> >> > {<br>
> >> > int origmaxp1 = p0 + 1;<br>
> >> > @@ -1392,11 +1395,45 @@<br>
> >> > for (int cp1 = p1; cp1 <= maxp1; cp1++)<br>
> >> > {<br>
> >> > if (!scenecutInternal(frames, p0, cp1, false))<br>
> >> > + {<br>
> >> > /* Any frame in between p0 and cur_p1 cannot be a real<br>
> >> scenecut. */<br>
> >> > for (int i = cp1; i > p0; i--)<br>
> >> > + {<br>
> >> > frames[i]->bScenecut = false;<br>
> >> > + frames[i]->bSceneChange = false;<br>
> >> > + }<br>
> >> > + }<br>
> >> > + else if (!scenecutInternal(frames,cp1 - 1, cp1, false))<br>
> >> > + {<br>
> >> > + /* Prevent false flashes. */<br>
> >> > + frames[cp1]->bSceneChange = false;<br>
> >> > + }<br>
> >> > + else<br>
> >> > + {<br>
> >> > + /* compute average satdcost of flashes of scenecuts to<br>
> >> confirm<br>
> >> > + * bwhether a scene transition is possible within the<br>
> >> segment. */<br>
> >> > + avgSatdCost += frames[cp1]->costEst[1][0];<br>
> >> > + cnt++;<br>
> >> > + }<br>
> >> > }<br>
> >> > -<br>
> >> > + /* Identify possible scene fluctuations by comparing the satd<br>
> >> cost of the flashes.<br>
> >> > + * This could denote the beginning or ending of scene<br>
> >> transitions.<br>
> >> > + * During a scene transition(fade in/fade outs), if fluctuate<br>
> >> remains false,<br>
> >> > + * then the scene had completed its transition or stabilized.<br>
> >> > + */<br>
> >> > + bool fluctuate = false;<br>
> >> > + if (avgSatdCost > 0)<br>
> >> > + {<br>
> >> > + avgSatdCost /= cnt;<br>
> >> > + for (int i= p0 ; i <= maxp1; i++)<br>
> >> > + {<br>
> >> > + if (abs(frames[i]->costEst[1][0] - avgSatdCost) > 0.1<br>
> >> * avgSatdCost)<br>
> >> > + {<br>
> >> > + fluctuate = true;<br>
> >> > + break;<br>
> >> > + }<br>
> >> > + }<br>
> >> > + }<br>
> >> > /* Where A-F are scenes: AAAAABBCCDDEEFFFFFF<br>
> >> > * If each of BB ... EE are shorter than (maxp1-p0), they are<br>
> >> > * detected as flashes and not considered scenecuts.<br>
> >> > @@ -1405,8 +1442,23 @@<br>
> >> > for (int cp0 = p0; cp0 <= maxp1; cp0++)<br>
> >> > {<br>
> >> > if (origmaxp1 > maxSearch || (cp0 < maxp1 &&<br>
> >> scenecutInternal(frames, cp0, maxp1, false)))<br>
> >> > + {<br>
> >> > /* If cur_p0 is the p0 of a scenecut, it cannot be the<br>
> >> p1 of a scenecut. */<br>
> >> > frames[cp0]->bScenecut = false;<br>
> >> > + /* Check for start of flash sequence for a new scene<br>
> >> transition segment. */<br>
> >> > + if (frames[cp0]->bSceneChange && !m_isSceneTransition<br>
> >> && scenecutInternal(frames, cp0, cp0 + 1, false) && fluctuate)<br>
> >> > + {<br>
> >> > + m_isSceneTransition = true;<br>
> >> > + }<br>
> >> > + else<br>
> >> > + frames[cp0]->bSceneChange = false;<br>
> >> > + }<br>
> >> > + }<br>
> >> > +<br>
> >> > + if (avgSatdCost == 0 || !fluctuate)<br>
> >> > + {<br>
> >> > + /* Signal end of scene transitioning */<br>
> >> > + m_isSceneTransition = false;<br>
> >> > }<br>
> >> > }<br>
> >> ><br>
> >> > diff -r 46152345eb6f -r f134920ac855 source/encoder/slicetype.h<br>
> >> > --- a/source/encoder/slicetype.h Mon Jul 20 17:18:54 2015 -0700<br>
> >> > +++ b/source/encoder/slicetype.h Wed Jul 22 00:29:26 2015 +0530<br>
> >> > @@ -127,7 +127,7 @@<br>
> >> > int m_numCoopSlices;<br>
> >> > int m_numRowsPerSlice;<br>
> >> > bool m_filled;<br>
> >> > -<br>
> >> > + bool m_isSceneTransition;<br>
> >> > Lookahead(x265_param *param, ThreadPool *pool);<br>
> >> ><br>
> >> > #if DETAILED_CU_STATS<br>
> >> > _______________________________________________<br>
> >> > x265-devel mailing list<br>
</div></div>> >> > <a href="mailto:x265-devel@videolan.org">x265-devel@videolan.org</a><br>
<span class="">> >> > <a href="https://mailman.videolan.org/listinfo/x265-devel" rel="noreferrer" target="_blank">https://mailman.videolan.org/listinfo/x265-devel</a><br>
> >><br>
> >> --<br>
> >> Steve Borho<br>
> >> _______________________________________________<br>
> >> x265-devel mailing list<br>
</span>> >> <a href="mailto:x265-devel@videolan.org">x265-devel@videolan.org</a><br>
<span class="">> >> <a href="https://mailman.videolan.org/listinfo/x265-devel" rel="noreferrer" target="_blank">https://mailman.videolan.org/listinfo/x265-devel</a><br>
> >><br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > x265-devel mailing list<br>
</span>> > <a href="mailto:x265-devel@videolan.org">x265-devel@videolan.org</a><br>
<span class="">> > <a href="https://mailman.videolan.org/listinfo/x265-devel" rel="noreferrer" target="_blank">https://mailman.videolan.org/listinfo/x265-devel</a><br>
> ><br>
> ><br>
<br>
> _______________________________________________<br>
> x265-devel mailing list<br>
</span>> <a href="mailto:x265-devel@videolan.org">x265-devel@videolan.org</a><br>
<span class="">> <a href="https://mailman.videolan.org/listinfo/x265-devel" rel="noreferrer" target="_blank">https://mailman.videolan.org/listinfo/x265-devel</a><br>
<br>
<br>
--<br>
Steve Borho<br>
_______________________________________________<br>
x265-devel mailing list<br>
</span><a href="mailto:x265-devel@videolan.org">x265-devel@videolan.org</a><br>
<a href="https://mailman.videolan.org/listinfo/x265-devel" rel="noreferrer" target="_blank">https://mailman.videolan.org/listinfo/x265-devel</a><br>
</blockquote></div><br></div></div>