[x265] Adaptove GoP sizes - not sure if its working
Roshantha Mendis
hrm506 at york.ac.uk
Tue Oct 6 23:49:13 CEST 2015
Okay thanks, I understand the versioning now.
But how do I checkout a specific version from the bitbucket repo ? e.g. If
I want to checkout 1.7+400
So I checked the latest version : x265 [info]: HEVC encoder version
1.7+512-f8b8ebdc5457
And I tested against the test.mp4 you have given (I had to convert it to
y4m first using ffmpeg, as x265 didn't recognise the format)
the the I-frames were at (POC): 0, 250, 500.
scenecut debug :
x265 [debug]: scene cut at 108 Icost:319097 Pcost:305903 ratio:0.0413
bias:0.2123 gop:108 (imb:1971 pmb:523)
x265 [debug]: scene cut at 151 Icost:1324811 Pcost:1318306 ratio:0.0049
bias:0.2692 gop:151 (imb:2470 pmb:24)
x265 [debug]: scene cut at 302 Icost:1505570 Pcost:1489614 ratio:0.0106
bias:0.1383 gop:52 (imb:2349 pmb:145)
x265 [debug]: scene cut at 397 Icost:3199040 Pcost:3189918 ratio:0.0029
bias:0.2639 gop:147 (imb:2344 pmb:150)
So I guess this is intentional as Deepthi explained: No I-frames are
inserted at scene-cuts because Pcost < Icost at these points. But Psize
becomes large at the scenecuts. Most of the time Pcost << Icost, so most of
the time, there won't be I-frames at scene changes.
I am interested in obtaining the behaviour seen in r436 (in the test
samples you gave) , but I using P frames instread of I frames make sense
from a compression point.
I've tried varying the --scenecut bias, but this does not seem to have any
effect, regardless of how high I increase it.
I did notice duplicate scenecut debug messages for another video I tested.
But I didn't investigate this much.
Thank you for your support
Rosh
On 6 October 2015 at 21:17, Xinyue Lu <maillist at 7086.in> wrote:
> Hello,
>
> I think they've found the issues in their patch. Maybe you can
> directly try their latest patch.
>
> And talking about version number, I have some ruby code that
> generating the same thing, which should be easier to read:
>
> ver = `hg log -r. --template "{latesttag}"`
> rev = `hg log -rdefault --template "{latesttagdistance}"`
> hash = `hg log -rdefault --template "{node|short}"`
> v = "#{ver}+#{rev}-#{hash}"
>
> Basically 512 is the distance to the latest tag, calculated by the hg
> command line above.
>
> Hope this helps.
>
> Xinyue
>
>
>
> On Tue, Oct 6, 2015 at 2:56 PM, Roshantha Mendis <hrm506 at york.ac.uk>
> wrote:
> > Hi,
> >
> > I would like to help test this bug further in my spare time.
> >
> > But I used to use SVN and still transitioning to mercurial/git.
> > how do I checkout/clone a particular version of x265 ? say 1.7+400 ?
> > Assuming 1.7+400 = 400 commits from 1.7 right ?
> > But when I check the actual revision number on the latest version, it
> says :
> >
> > $ hg identify -nibtB
> > f8b8ebdc5457 11036 default tip
> >
> > How does 11036 translate to 1.7+512 ??
> >
> > thanks
> > Rosh
> >
> >
> >
> > On 6 October 2015 at 12:38, Deepthi Nandakumar
> > <deepthi at multicorewareinc.com> wrote:
> >>
> >> Thank you for this test case.
> >>
> >> We have identified a series of bugs on this, and are focused on closing
> >> this asap and tagging 1.8. I have sent one patch - would be good to get
> >> feedback.
> >>
> >> On Tue, Oct 6, 2015 at 7:10 AM, Xinyue Lu <maillist at 7086.in> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I think maybe it's better for me to leave comment in this thread.
> >>>
> >>> Here you are, including my builds, test video clips and the results.
> >>>
> >>> https://www.dropbox.com/s/6axpqr9wozeeqsu/x265%20patch%20test.tgz?dl=0
> >>>
> >>> It should be clear to see that in rev437, I frames are inserted at 0,
> >>> 250, 500, while in rev436, they are inserted at 0, 37, 77, 113, 151,
> >>> 302, 350, 397 and 569, where most of them are real scene cut.
> >>>
> >>> Disclaimer: the video clips are commercials that are randomly picked,
> >>> and if violate any copyright, will be removed.
> >>>
> >>> Thanks.
> >>>
> >>> On Mon, Oct 5, 2015 at 9:27 PM, Deepthi Nandakumar
> >>> <deepthi at multicorewareinc.com> wrote:
> >>> > Sorry - in b) below, it should be icost is greater than pcost.
> >>> >
> >>> > On Tue, Oct 6, 2015 at 6:56 AM, Deepthi Nandakumar
> >>> > <deepthi at multicorewareinc.com> wrote:
> >>> >>
> >>> >> We recently modified the scenecut logic slightly. There are 2 levels
> >>> >> of
> >>> >> scenecuts detected
> >>> >>
> >>> >> a) One is where the I-frame cost is less than P-frame cost (with
> bias
> >>> >> factor). This inserts an I-frame and is the scenecut everyone is
> >>> >> familiar
> >>> >> with.
> >>> >>
> >>> >> b) This next case is also logged as a scenecut - while the slicetype
> >>> >> remains P (because the icost is lesser than pcost), ratecontrol
> >>> >> realises
> >>> >> that there's some scene transition going on, and so uses extra logic
> >>> >> to
> >>> >> retain quality. Without this, we found that there was blocking at
> the
> >>> >> start
> >>> >> of a new scene, fade-ins/fade-outs etc.
> >>> >>
> >>> >>
> >>> >>
> >>> >> On Tue, Oct 6, 2015 at 1:54 AM, Roshantha Mendis <hrm506 at york.ac.uk
> >
> >>> >> wrote:
> >>> >>>
> >>> >>> Hi Deepthi,
> >>> >>>
> >>> >>> I'm very sorry to open up this thread again after so long, but I
> >>> >>> tried
> >>> >>> enabling the debug log as you've said. And I get scenecut
> >>> >>> notifications such
> >>> >>> as :
> >>> >>>
> >>> >>> x265 [debug]: scene cut at 102 Icost:160729 Pcost:153370
> ratio:0.0458
> >>> >>> bias:5394.1909 gop:102 (imb:983 pmb:479)
> >>> >>> x265 [debug]: scene cut at 161 Icost:567007 Pcost:558367
> ratio:0.0152
> >>> >>> bias:7230.2905 gop:161 (imb:1015 pmb:447)
> >>> >>> x265 [debug]: scene cut at 286 Icost:329587 Pcost:316290
> ratio:0.0403
> >>> >>> bias:3340.2490 gop:36 (imb:1018 pmb:444)
> >>> >>> ...
> >>> >>> ...
> >>> >>> ...
> >>> >>>
> >>> >>>
> >>> >>> Each of the frames above notified by the debug (102, 161, 286..)
> are
> >>> >>> P-frames.
> >>> >>>
> >>> >>> The debug is coming from slicetype::scenecutInternal, and 'res' is
> >>> >>> returned as follows:
> >>> >>>
> >>> >>> bool res = pcost >= (1.0 - bias) * icost;
> >>> >>> if (res && bRealScenecut)
> >>> >>> {
> >>> >>> int imb = frame->intraMbs[p1 - p0];
> >>> >>> int pmb = m_8x8Blocks - imb;
> >>> >>> x265_log(m_param, X265_LOG_DEBUG, "scene cut at %d Icost:%d
> >>> >>> Pcost:%d ratio:%.4f bias:%.4f gop:%d (imb:%d pmb:%d)\n",
> >>> >>> frame->frameNum, icost, pcost, 1. - (double)pcost
> /
> >>> >>> icost, bias, gopSize, imb, pmb);
> >>> >>> }
> >>> >>> return res;
> >>> >>>
> >>> >>> And in slicetype::slicetypeAnalyse, scenecut() gets called and it
> >>> >>> does
> >>> >>> this :
> >>> >>>
> >>> >>> if (m_param->bFrameAdaptive)
> >>> >>> {
> >>> >>> bool isScenecut = scenecut(frames, 0, 1, true,
> >>> >>> origNumFrames);
> >>> >>> /* When scenecut threshold is set, use scenecut detection
> for
> >>> >>> I
> >>> >>> frame placements */
> >>> >>> if (!m_param->scenecutThreshold && isScenecut)
> >>> >>> {
> >>> >>> frames[1]->sliceType = X265_TYPE_I;
> >>> >>> return;
> >>> >>> }
> >>> >>> }
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> So I am very confused, technically when the scene-cut debug appears
> >>> >>> it
> >>> >>> means there should be an I-frame at that point right ?? (by
> >>> >>> calculating the
> >>> >>> 'res' bool : res = pcost >= (1.0 - bias) * icost;). But this does
> not
> >>> >>> seem
> >>> >>> the case.
> >>> >>>
> >>> >>>
> >>> >>> I am using the following :
> >>> >>>
> >>> >>> ../HEVC_x265/x265/build/linux/x265 --input vid_576p_extracted.y4m
> >>> >>> --rc-lookahead 250 --scenecut 1000000 -i 9 -F 0 -I 250 -o vid.x265
> >>> >>> --csv
> >>> >>> temp.csv --csv-log-level 2 --log-level 4 --b-adapt 2 --bframes 4
> >>> >>> --no-weightp --no-weightb --no-open-gop --b-intra --ref 1 --frames
> >>> >>> 10000000
> >>> >>> > temp_debug.log 2>&1
> >>> >>>
> >>> >>>
> >>> >>> Many thanks
> >>> >>> Rosh
> >>> >>>
> >>> >>>
> >>> >>> On 24 August 2015 at 05:27, Deepthi Nandakumar
> >>> >>> <deepthi at multicorewareinc.com> wrote:
> >>> >>>>
> >>> >>>> If you need more details, you can look through
> >>> >>>> slicetype.cpp::scenecut.
> >>> >>>> We recently changed the algorithm slightly.
> >>> >>>>
> >>> >>>> a) A scenechange flag is set based purely on frame content changes
> >>> >>>> and
> >>> >>>> scenecut threshold.
> >>> >>>> b) At the time of slicetype decision, for all "scene changed"
> frames
> >>> >>>> the
> >>> >>>> distance from last keyframe is taken into account. The further
> away
> >>> >>>> it is,
> >>> >>>> the more likely the current "scene changed" frame will be an I
> >>> >>>> frame. If
> >>> >>>> this condition is not satisfied, it may become a P frame (with a
> >>> >>>> likely
> >>> >>>> large framesize).
> >>> >>>>
> >>> >>>> You can log how these flags change, and see how the adaptive GOP
> is
> >>> >>>> determined for each video.
> >>> >>>>
> >>> >>>> On Sun, Aug 23, 2015 at 9:39 PM, Steve Borho <steve at borho.org>
> >>> >>>> wrote:
> >>> >>>>>
> >>> >>>>> On 08/22, Roshantha Mendis wrote:
> >>> >>>>> > Thanks Steve and Deepthi, for explaining things and for that
> >>> >>>>> > link.
> >>> >>>>> >
> >>> >>>>> > I understand now, that the encoder will always attempt to use
> >>> >>>>> > P-frames at
> >>> >>>>> > scene-cuts, whenever possible. Is this to improve on
> compression
> >>> >>>>> > efficiency
> >>> >>>>> > ?
> >>> >>>>> >
> >>> >>>>> > However in the docs it says :
> >>> >>>>> > --scenecut <integer>, --no-scenecut
> >>> >>>>> >
> >>> >>>>> > How aggressively I-frames need to be inserted. *The higher the
> >>> >>>>> > threshold
> >>> >>>>> > value, the more aggressive the I-frame placement*. --scenecut
> >>> >>>>> >
> >>> >>>>> >
> >>> >>>>> > <
> https://x265.readthedocs.org/en/default/cli.html#cmdoption--scenecut> 0 or
> >>> >>>>> > --no-scenecut
> >>> >>>>> >
> >>> >>>>> >
> >>> >>>>> > <
> https://x265.readthedocs.org/en/default/cli.html#cmdoption--no-scenecut>
> >>> >>>>> > disables adaptive I frame placement. Default 40
> >>> >>>>> >
> >>> >>>>> > So I thought , increasing this *'--scenecut <int>'* param would
> >>> >>>>> > cause
> >>> >>>>> > the
> >>> >>>>> > encoder to insert I-frames instead of P-frames. is there an
> upper
> >>> >>>>> > limit to
> >>> >>>>> > the scenecut param ?
> >>> >>>>> >
> >>> >>>>> > and on the x265 website (https://x265.com/about-x265/) it
> says :
> >>> >>>>> > Scenecut:
> >>> >>>>> > Insert I-frames at scene changes
> >>> >>>>> > which is why I thought the encoder would insert I-frames rather
> >>> >>>>> > than
> >>> >>>>> > P-frames. Initially I was going to use x264 to get GoP size
> >>> >>>>> > distribution
> >>> >>>>> > results. But when I saw those lines in the docs/website I
> thought
> >>> >>>>> > of
> >>> >>>>> > trying
> >>> >>>>> > out x265 instead.
> >>> >>>>>
> >>> >>>>> That is what the scenecut bias does, but increasing --scenecut
> does
> >>> >>>>> not
> >>> >>>>> necessarily make the encoder always use I frames for scene-cuts.
> >>> >>>>> The
> >>> >>>>> keyframe adaption function uses a bias that increases the
> >>> >>>>> likelyhood of
> >>> >>>>> an I (vs a P) the closer it gets to the max keyframe interval.
> The
> >>> >>>>> --scenecut bias adds to that bias, making early keyframes more
> >>> >>>>> likely.
> >>> >>>>>
> >>> >>>>> FWIW: x264 has almost the exact same logic
> >>> >>>>>
> >>> >>>>> > For, the purpose of presenting results in an academic paper -
> do
> >>> >>>>> > you
> >>> >>>>> > think
> >>> >>>>> > calculating a GoP size as distance between : [ I-SLICE <-->
> I/PP
> >>> >>>>> > SLICE] is
> >>> >>>>> > correct ?
> >>> >>>>>
> >>> >>>>> GOP size is generally the distance between keyframes.
> >>> >>>>>
> >>> >>>>> > Finally - with respect to closed-GOPs (IDR is being used) - Is
> it
> >>> >>>>> > right to
> >>> >>>>> > say that in this case, GoP size (I to I distance), will always
> be
> >>> >>>>> > fixed at
> >>> >>>>> > the --keyint inverval : no variation, not even slight. Is this
> >>> >>>>> > because with
> >>> >>>>> > closed-gops it is not possible to have adaptive GoPs ? (i.e. it
> >>> >>>>> > doesn't
> >>> >>>>> > specify in the standards) or just the x265 encoder doesn't
> >>> >>>>> > support it
> >>> >>>>> > ?
> >>> >>>>>
> >>> >>>>> no, closed-GOP only means that at the keyframes the decoded
> picture
> >>> >>>>> buffer is cleared, so pictures decoded after the keyframe cannot
> >>> >>>>> use
> >>> >>>>> any
> >>> >>>>> references that were coded before the keyframe. It has nothing to
> >>> >>>>> do
> >>> >>>>> with keyframe cadence.
> >>> >>>>>
> >>> >>>>> fixed keyframe interval is achieved by --no-scenecut
> >>> >>>>>
> >>> >>>>> --
> >>> >>>>> Steve Borho
> >>> >>>>> _______________________________________________
> >>> >>>>> x265-devel mailing list
> >>> >>>>> x265-devel at videolan.org
> >>> >>>>> https://mailman.videolan.org/listinfo/x265-devel
> >>> >>>>
> >>> >>>>
> >>> >>>>
> >>> >>>> _______________________________________________
> >>> >>>> x265-devel mailing list
> >>> >>>> x265-devel at videolan.org
> >>> >>>> https://mailman.videolan.org/listinfo/x265-devel
> >>> >>>>
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> --
> >>> >>> Rosh Mendis
> >>> >>> Research Student - EngD in LSCITS
> >>> >>> Dept. of Computer Science
> >>> >>> University of York
> >>> >>>
> >>> >>> Disclaimer: http://www.york.ac.uk/docs/disclaimer/email.htm
> >>> >>>
> >>> >>> _______________________________________________
> >>> >>> x265-devel mailing list
> >>> >>> x265-devel at videolan.org
> >>> >>> https://mailman.videolan.org/listinfo/x265-devel
> >>> >>>
> >>> >>
> >>> >
> >>> >
> >>> > _______________________________________________
> >>> > x265-devel mailing list
> >>> > x265-devel at videolan.org
> >>> > https://mailman.videolan.org/listinfo/x265-devel
> >>> >
> >>> _______________________________________________
> >>> x265-devel mailing list
> >>> x265-devel at videolan.org
> >>> https://mailman.videolan.org/listinfo/x265-devel
> >>
> >>
> >>
> >> _______________________________________________
> >> x265-devel mailing list
> >> x265-devel at videolan.org
> >> https://mailman.videolan.org/listinfo/x265-devel
> >>
> >
> >
> >
> > --
> > Rosh Mendis
> > Research Student - EngD in LSCITS
> > Dept. of Computer Science
> > University of York
> >
> > Disclaimer: http://www.york.ac.uk/docs/disclaimer/email.htm
> >
> > _______________________________________________
> > x265-devel mailing list
> > x265-devel at videolan.org
> > https://mailman.videolan.org/listinfo/x265-devel
> >
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
--
Rosh Mendis
Research Student - EngD in LSCITS
Dept. of Computer Science
University of York
Disclaimer: http://www.york.ac.uk/docs/disclaimer/email.htm
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20151006/a7adfc6a/attachment-0001.html>
More information about the x265-devel
mailing list