[x265] [PATCH] slicetype: Modify Scenecut algorithm to detect scene transition points
Aarthi Priya Thirumalai
aarthi at multicorewareinc.com
Thu Jul 23 20:47:43 CEST 2015
On Wed, Jul 22, 2015 at 11:08 PM, Steve Borho <steve at borho.org> wrote:
> On 07/22, Aarthi Priya Thirumalai wrote:
> > Currently during scene change transitions, esp slow gradual
> > fadeins/fadeouts every frame changes vastly in luma intensity and hence
> > passes as a scenecut but these are treated
> > as a series of flashes in scenecut() . So, none of the frames gets marked
> > as scencut during such transitions.
> > It would be better if we can signal the first frame of such transition
> > flashes as sceneChange for RateControl. so that the rate control can set
> a
> > minimum qp for such frames.
> > This would help the bitrate spikes at fade ins where the qp can be really
> > low.
>
> weightp in the lookahead doesn't help for fades? or is this a problem in
> presets with weightp disabled?
>
weightp doesn't help for fades. happens even when weightp is enabled.
>
> > The patch mainly fixes the place in scenecut() where the flash detection
> > happens and the first frame in a series of flash is signaled as
> > sceneChange.
> > Changing bias or modifying scenecutInterval doesn't help here as those
> > frames are already ,marked as scenecuts from those function.
> > Only scencut() flash detection part reverts the flashes as non-scenecuts
> > which we are changing.
> >
> > the slicetype decision is not changed as I am signalling these scene
> > changes with a different flag that Ratecontrol would be using.
>
> this problem of fades being detected as flashes is probably more
> prevelant when the keyframe distance approaches the limit, the bias
> ensures this. it seems like both sides of it need to be addressed.
>
actually even when the keyframe interval is not closer to the transition,
they get detected
as flashes only. for each frame in the fadein, the satdcosts are strong
wnough to overcome the bias and get detected as scenecut. I experimented
with constant bias throughout (remove keyframe effect from bias
calculation) yet the fades are getting detected as a series of flashes.
Hence I am trying to signal the first frame in a series of flashes for rate
control rather than the last frame as currently signalled by scenecut().
since I am using a different flag,current slice type decisions are not
affected.
> > On Wed, Jul 22, 2015 at 10:10 PM, Deepthi Nandakumar <
> > deepthi at multicorewareinc.com> wrote:
> >
> > > The intention was to separate slicetype decision from scene change
> > > detection, where scene change detection would also account for
> transitions
> > > (like fade-ins, fade-outs etc), in addition to the abrupt scene change.
> > >
> > > It is not clear to me how this is being done without modifying
> > > scenecutInternal.
> > >
> > > On Wed, Jul 22, 2015 at 9:59 PM, Steve Borho <steve at borho.org> wrote:
> > >
> > >> On 07/22, aarthi at multicorewareinc.com wrote:
> > >> > # HG changeset patch
> > >> > # User Aarthi Thirumalai
> > >> > # Date 1437505166 -19800
> > >> > # Wed Jul 22 00:29:26 2015 +0530
> > >> > # Node ID f134920ac855af45f052ccd6d3c239d39940a100
> > >> > # Parent 46152345eb6ff261fd90272f7a0712300d6324c0
> > >> > slicetype: Modify Scenecut algorithm to detect scene transition
> points
> > >> > to improve Rate Control.
> > >> >
> > >> > identify scene trasitions, fade-ins, fadeouts, sceneCuts and signal
> the
> > >> flag bSceneChange
> > >> > in Lowres structure. This flag will be used by RateControl to adjust
> > >> the wps during scene cuts.
> > >> > No changes in sliceType decisions or I frame placement.
> > >>
> > >> I could nitpick a few coding style issues, but my main question is
> > >> whether you tried disabling the keyframe distance bias during flash
> > >> detection and arrived at this approach, or whether this is an
> orthogonal
> > >> change?
> > >>
> > >> > diff -r 46152345eb6f -r f134920ac855 source/common/lowres.cpp
> > >> > --- a/source/common/lowres.cpp Mon Jul 20 17:18:54 2015 -0700
> > >> > +++ b/source/common/lowres.cpp Wed Jul 22 00:29:26 2015 +0530
> > >> > @@ -127,6 +127,7 @@
> > >> > {
> > >> > bLastMiniGopBFrame = false;
> > >> > bScenecut = true; // could be a scene-cut, until ruled out by
> > >> flash detection
> > >> > + bSceneChange = true;
> > >> > bKeyframe = false; // Not a keyframe unless identified by
> lookahead
> > >> > frameNum = poc;
> > >> > leadingBframes = 0;
> > >> > diff -r 46152345eb6f -r f134920ac855 source/common/lowres.h
> > >> > --- a/source/common/lowres.h Mon Jul 20 17:18:54 2015 -0700
> > >> > +++ b/source/common/lowres.h Wed Jul 22 00:29:26 2015 +0530
> > >> > @@ -115,6 +115,7 @@
> > >> > int leadingBframes; // number of leading B frames for P
> or I
> > >> >
> > >> > bool bScenecut; // Set to false if the frame cannot
> > >> possibly be part of a real scenecut.
> > >> > + bool bSceneChange;
> > >> > bool bKeyframe;
> > >> > bool bLastMiniGopBFrame;
> > >> >
> > >> > diff -r 46152345eb6f -r f134920ac855 source/encoder/slicetype.cpp
> > >> > --- a/source/encoder/slicetype.cpp Mon Jul 20 17:18:54 2015 -0700
> > >> > +++ b/source/encoder/slicetype.cpp Wed Jul 22 00:29:26 2015 +0530
> > >> > @@ -462,6 +462,7 @@
> > >> > m_pool = pool;
> > >> >
> > >> > m_lastNonB = NULL;
> > >> > + m_isSceneTransition = false;
> > >> > m_scratch = NULL;
> > >> > m_tld = NULL;
> > >> > m_filled = false;
> > >> > @@ -1375,6 +1376,8 @@
> > >> > bool Lookahead::scenecut(Lowres **frames, int p0, int p1, bool
> > >> bRealScenecut, int numFrames, int maxSearch)
> > >> > {
> > >> > /* Only do analysis during a normal scenecut check. */
> > >> > + int64_t avgSatdCost = 0;
> > >> > + int cnt = 0;
> > >> > if (bRealScenecut && m_param->bframes)
> > >> > {
> > >> > int origmaxp1 = p0 + 1;
> > >> > @@ -1392,11 +1395,45 @@
> > >> > for (int cp1 = p1; cp1 <= maxp1; cp1++)
> > >> > {
> > >> > if (!scenecutInternal(frames, p0, cp1, false))
> > >> > + {
> > >> > /* Any frame in between p0 and cur_p1 cannot be a
> real
> > >> scenecut. */
> > >> > for (int i = cp1; i > p0; i--)
> > >> > + {
> > >> > frames[i]->bScenecut = false;
> > >> > + frames[i]->bSceneChange = false;
> > >> > + }
> > >> > + }
> > >> > + else if (!scenecutInternal(frames,cp1 - 1, cp1, false))
> > >> > + {
> > >> > + /* Prevent false flashes. */
> > >> > + frames[cp1]->bSceneChange = false;
> > >> > + }
> > >> > + else
> > >> > + {
> > >> > + /* compute average satdcost of flashes of
> scenecuts to
> > >> confirm
> > >> > + * bwhether a scene transition is possible within
> the
> > >> segment. */
> > >> > + avgSatdCost += frames[cp1]->costEst[1][0];
> > >> > + cnt++;
> > >> > + }
> > >> > }
> > >> > -
> > >> > + /* Identify possible scene fluctuations by comparing the
> satd
> > >> cost of the flashes.
> > >> > + * This could denote the beginning or ending of scene
> > >> transitions.
> > >> > + * During a scene transition(fade in/fade outs), if
> fluctuate
> > >> remains false,
> > >> > + * then the scene had completed its transition or
> stabilized.
> > >> > + */
> > >> > + bool fluctuate = false;
> > >> > + if (avgSatdCost > 0)
> > >> > + {
> > >> > + avgSatdCost /= cnt;
> > >> > + for (int i= p0 ; i <= maxp1; i++)
> > >> > + {
> > >> > + if (abs(frames[i]->costEst[1][0] - avgSatdCost) >
> 0.1
> > >> * avgSatdCost)
> > >> > + {
> > >> > + fluctuate = true;
> > >> > + break;
> > >> > + }
> > >> > + }
> > >> > + }
> > >> > /* Where A-F are scenes: AAAAABBCCDDEEFFFFFF
> > >> > * If each of BB ... EE are shorter than (maxp1-p0), they
> are
> > >> > * detected as flashes and not considered scenecuts.
> > >> > @@ -1405,8 +1442,23 @@
> > >> > for (int cp0 = p0; cp0 <= maxp1; cp0++)
> > >> > {
> > >> > if (origmaxp1 > maxSearch || (cp0 < maxp1 &&
> > >> scenecutInternal(frames, cp0, maxp1, false)))
> > >> > + {
> > >> > /* If cur_p0 is the p0 of a scenecut, it cannot be
> the
> > >> p1 of a scenecut. */
> > >> > frames[cp0]->bScenecut = false;
> > >> > + /* Check for start of flash sequence for a new
> scene
> > >> transition segment. */
> > >> > + if (frames[cp0]->bSceneChange &&
> !m_isSceneTransition
> > >> && scenecutInternal(frames, cp0, cp0 + 1, false) && fluctuate)
> > >> > + {
> > >> > + m_isSceneTransition = true;
> > >> > + }
> > >> > + else
> > >> > + frames[cp0]->bSceneChange = false;
> > >> > + }
> > >> > + }
> > >> > +
> > >> > + if (avgSatdCost == 0 || !fluctuate)
> > >> > + {
> > >> > + /* Signal end of scene transitioning */
> > >> > + m_isSceneTransition = false;
> > >> > }
> > >> > }
> > >> >
> > >> > diff -r 46152345eb6f -r f134920ac855 source/encoder/slicetype.h
> > >> > --- a/source/encoder/slicetype.h Mon Jul 20 17:18:54 2015 -0700
> > >> > +++ b/source/encoder/slicetype.h Wed Jul 22 00:29:26 2015 +0530
> > >> > @@ -127,7 +127,7 @@
> > >> > int m_numCoopSlices;
> > >> > int m_numRowsPerSlice;
> > >> > bool m_filled;
> > >> > -
> > >> > + bool m_isSceneTransition;
> > >> > Lookahead(x265_param *param, ThreadPool *pool);
> > >> >
> > >> > #if DETAILED_CU_STATS
> > >> > _______________________________________________
> > >> > x265-devel mailing list
> > >> > x265-devel at videolan.org
> > >> > https://mailman.videolan.org/listinfo/x265-devel
> > >>
> > >> --
> > >> Steve Borho
> > >> _______________________________________________
> > >> x265-devel mailing list
> > >> x265-devel at videolan.org
> > >> https://mailman.videolan.org/listinfo/x265-devel
> > >>
> > >
> > >
> > > _______________________________________________
> > > x265-devel mailing list
> > > x265-devel at videolan.org
> > > https://mailman.videolan.org/listinfo/x265-devel
> > >
> > >
>
> > _______________________________________________
> > x265-devel mailing list
> > x265-devel at videolan.org
> > https://mailman.videolan.org/listinfo/x265-devel
>
>
> --
> Steve Borho
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20150724/2e115481/attachment-0001.html>
More information about the x265-devel
mailing list