[x265] [PATCH] slicetype: Modify Scenecut algorithm to detect scene transition points
Steve Borho
steve at borho.org
Wed Jul 22 19:38:28 CEST 2015
On 07/22, Aarthi Priya Thirumalai wrote:
> Currently during scene change transitions, esp slow gradual
> fadeins/fadeouts every frame changes vastly in luma intensity and hence
> passes as a scenecut but these are treated
> as a series of flashes in scenecut() . So, none of the frames gets marked
> as scencut during such transitions.
> It would be better if we can signal the first frame of such transition
> flashes as sceneChange for RateControl. so that the rate control can set a
> minimum qp for such frames.
> This would help the bitrate spikes at fade ins where the qp can be really
> low.
weightp in the lookahead doesn't help for fades? or is this a problem in
presets with weightp disabled?
> The patch mainly fixes the place in scenecut() where the flash detection
> happens and the first frame in a series of flash is signaled as
> sceneChange.
> Changing bias or modifying scenecutInterval doesn't help here as those
> frames are already ,marked as scenecuts from those function.
> Only scencut() flash detection part reverts the flashes as non-scenecuts
> which we are changing.
>
> the slicetype decision is not changed as I am signalling these scene
> changes with a different flag that Ratecontrol would be using.
this problem of fades being detected as flashes is probably more
prevelant when the keyframe distance approaches the limit, the bias
ensures this. it seems like both sides of it need to be addressed.
> On Wed, Jul 22, 2015 at 10:10 PM, Deepthi Nandakumar <
> deepthi at multicorewareinc.com> wrote:
>
> > The intention was to separate slicetype decision from scene change
> > detection, where scene change detection would also account for transitions
> > (like fade-ins, fade-outs etc), in addition to the abrupt scene change.
> >
> > It is not clear to me how this is being done without modifying
> > scenecutInternal.
> >
> > On Wed, Jul 22, 2015 at 9:59 PM, Steve Borho <steve at borho.org> wrote:
> >
> >> On 07/22, aarthi at multicorewareinc.com wrote:
> >> > # HG changeset patch
> >> > # User Aarthi Thirumalai
> >> > # Date 1437505166 -19800
> >> > # Wed Jul 22 00:29:26 2015 +0530
> >> > # Node ID f134920ac855af45f052ccd6d3c239d39940a100
> >> > # Parent 46152345eb6ff261fd90272f7a0712300d6324c0
> >> > slicetype: Modify Scenecut algorithm to detect scene transition points
> >> > to improve Rate Control.
> >> >
> >> > identify scene trasitions, fade-ins, fadeouts, sceneCuts and signal the
> >> flag bSceneChange
> >> > in Lowres structure. This flag will be used by RateControl to adjust
> >> the wps during scene cuts.
> >> > No changes in sliceType decisions or I frame placement.
> >>
> >> I could nitpick a few coding style issues, but my main question is
> >> whether you tried disabling the keyframe distance bias during flash
> >> detection and arrived at this approach, or whether this is an orthogonal
> >> change?
> >>
> >> > diff -r 46152345eb6f -r f134920ac855 source/common/lowres.cpp
> >> > --- a/source/common/lowres.cpp Mon Jul 20 17:18:54 2015 -0700
> >> > +++ b/source/common/lowres.cpp Wed Jul 22 00:29:26 2015 +0530
> >> > @@ -127,6 +127,7 @@
> >> > {
> >> > bLastMiniGopBFrame = false;
> >> > bScenecut = true; // could be a scene-cut, until ruled out by
> >> flash detection
> >> > + bSceneChange = true;
> >> > bKeyframe = false; // Not a keyframe unless identified by lookahead
> >> > frameNum = poc;
> >> > leadingBframes = 0;
> >> > diff -r 46152345eb6f -r f134920ac855 source/common/lowres.h
> >> > --- a/source/common/lowres.h Mon Jul 20 17:18:54 2015 -0700
> >> > +++ b/source/common/lowres.h Wed Jul 22 00:29:26 2015 +0530
> >> > @@ -115,6 +115,7 @@
> >> > int leadingBframes; // number of leading B frames for P or I
> >> >
> >> > bool bScenecut; // Set to false if the frame cannot
> >> possibly be part of a real scenecut.
> >> > + bool bSceneChange;
> >> > bool bKeyframe;
> >> > bool bLastMiniGopBFrame;
> >> >
> >> > diff -r 46152345eb6f -r f134920ac855 source/encoder/slicetype.cpp
> >> > --- a/source/encoder/slicetype.cpp Mon Jul 20 17:18:54 2015 -0700
> >> > +++ b/source/encoder/slicetype.cpp Wed Jul 22 00:29:26 2015 +0530
> >> > @@ -462,6 +462,7 @@
> >> > m_pool = pool;
> >> >
> >> > m_lastNonB = NULL;
> >> > + m_isSceneTransition = false;
> >> > m_scratch = NULL;
> >> > m_tld = NULL;
> >> > m_filled = false;
> >> > @@ -1375,6 +1376,8 @@
> >> > bool Lookahead::scenecut(Lowres **frames, int p0, int p1, bool
> >> bRealScenecut, int numFrames, int maxSearch)
> >> > {
> >> > /* Only do analysis during a normal scenecut check. */
> >> > + int64_t avgSatdCost = 0;
> >> > + int cnt = 0;
> >> > if (bRealScenecut && m_param->bframes)
> >> > {
> >> > int origmaxp1 = p0 + 1;
> >> > @@ -1392,11 +1395,45 @@
> >> > for (int cp1 = p1; cp1 <= maxp1; cp1++)
> >> > {
> >> > if (!scenecutInternal(frames, p0, cp1, false))
> >> > + {
> >> > /* Any frame in between p0 and cur_p1 cannot be a real
> >> scenecut. */
> >> > for (int i = cp1; i > p0; i--)
> >> > + {
> >> > frames[i]->bScenecut = false;
> >> > + frames[i]->bSceneChange = false;
> >> > + }
> >> > + }
> >> > + else if (!scenecutInternal(frames,cp1 - 1, cp1, false))
> >> > + {
> >> > + /* Prevent false flashes. */
> >> > + frames[cp1]->bSceneChange = false;
> >> > + }
> >> > + else
> >> > + {
> >> > + /* compute average satdcost of flashes of scenecuts to
> >> confirm
> >> > + * bwhether a scene transition is possible within the
> >> segment. */
> >> > + avgSatdCost += frames[cp1]->costEst[1][0];
> >> > + cnt++;
> >> > + }
> >> > }
> >> > -
> >> > + /* Identify possible scene fluctuations by comparing the satd
> >> cost of the flashes.
> >> > + * This could denote the beginning or ending of scene
> >> transitions.
> >> > + * During a scene transition(fade in/fade outs), if fluctuate
> >> remains false,
> >> > + * then the scene had completed its transition or stabilized.
> >> > + */
> >> > + bool fluctuate = false;
> >> > + if (avgSatdCost > 0)
> >> > + {
> >> > + avgSatdCost /= cnt;
> >> > + for (int i= p0 ; i <= maxp1; i++)
> >> > + {
> >> > + if (abs(frames[i]->costEst[1][0] - avgSatdCost) > 0.1
> >> * avgSatdCost)
> >> > + {
> >> > + fluctuate = true;
> >> > + break;
> >> > + }
> >> > + }
> >> > + }
> >> > /* Where A-F are scenes: AAAAABBCCDDEEFFFFFF
> >> > * If each of BB ... EE are shorter than (maxp1-p0), they are
> >> > * detected as flashes and not considered scenecuts.
> >> > @@ -1405,8 +1442,23 @@
> >> > for (int cp0 = p0; cp0 <= maxp1; cp0++)
> >> > {
> >> > if (origmaxp1 > maxSearch || (cp0 < maxp1 &&
> >> scenecutInternal(frames, cp0, maxp1, false)))
> >> > + {
> >> > /* If cur_p0 is the p0 of a scenecut, it cannot be the
> >> p1 of a scenecut. */
> >> > frames[cp0]->bScenecut = false;
> >> > + /* Check for start of flash sequence for a new scene
> >> transition segment. */
> >> > + if (frames[cp0]->bSceneChange && !m_isSceneTransition
> >> && scenecutInternal(frames, cp0, cp0 + 1, false) && fluctuate)
> >> > + {
> >> > + m_isSceneTransition = true;
> >> > + }
> >> > + else
> >> > + frames[cp0]->bSceneChange = false;
> >> > + }
> >> > + }
> >> > +
> >> > + if (avgSatdCost == 0 || !fluctuate)
> >> > + {
> >> > + /* Signal end of scene transitioning */
> >> > + m_isSceneTransition = false;
> >> > }
> >> > }
> >> >
> >> > diff -r 46152345eb6f -r f134920ac855 source/encoder/slicetype.h
> >> > --- a/source/encoder/slicetype.h Mon Jul 20 17:18:54 2015 -0700
> >> > +++ b/source/encoder/slicetype.h Wed Jul 22 00:29:26 2015 +0530
> >> > @@ -127,7 +127,7 @@
> >> > int m_numCoopSlices;
> >> > int m_numRowsPerSlice;
> >> > bool m_filled;
> >> > -
> >> > + bool m_isSceneTransition;
> >> > Lookahead(x265_param *param, ThreadPool *pool);
> >> >
> >> > #if DETAILED_CU_STATS
> >> > _______________________________________________
> >> > x265-devel mailing list
> >> > x265-devel at videolan.org
> >> > https://mailman.videolan.org/listinfo/x265-devel
> >>
> >> --
> >> Steve Borho
> >> _______________________________________________
> >> x265-devel mailing list
> >> x265-devel at videolan.org
> >> https://mailman.videolan.org/listinfo/x265-devel
> >>
> >
> >
> > _______________________________________________
> > x265-devel mailing list
> > x265-devel at videolan.org
> > https://mailman.videolan.org/listinfo/x265-devel
> >
> >
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
--
Steve Borho
More information about the x265-devel
mailing list