[vlc-devel] [PATCH] Fixed incorrect backward/forward reference ordering in vaapi deinterlacing

[Steve Lhomme]

On Tue, Jul 4, 2017 at 1:32 PM, Oliver Collyer <ovcollyer at mac.com> wrote:
>
> On 4 Jul 2017, at 13:57, Steve Lhomme <robux4 at gmail.com> wrote:
>
> On Tue, Jul 4, 2017 at 11:52 AM, Oliver Collyer <ovcollyer at mac.com> wrote:
>
>
>
> On 27 Jun 2017, at 23:26, Thomas Guillem <thomas at gllm.fr> wrote:
>
>
>
> On Tue, Jun 27, 2017, at 21:21, Oliver Collyer wrote:
>
> On 27 Jun 2017, at 16:25, Victorien Le Couviour--Tuffet
> <victorien.lecouviour.tuffet at gmail.com> wrote:
>
> On Tue, Jun 27, 2017 at 03:58:48PM +0300, Oliver Collyer wrote:
>
> After testing the Motion Adaptive deinterlacing mode and finding it produced
> lots of flicker/judder and not much deinterlacing I traced the error back to
> the below.
>
> Although it seems a little counter-intuitive, the "forward_refs" are
> actually the frames older than the current one, and the "backward_refs" are
> those later.
>
> In addition, the forward_refs (previously the backward_refs) have to be
> ordered such that [0] is most recent, [1] older, etc; before this patch it
> was the opposite.
>
> Just to be 100% sure, I have also cross-referenced this approach with how
> FFmpeg does it:
>
> https://github.com/FFmpeg/FFmpeg/blob/master/libavfilter/vf_deinterlace_vaapi.c
> <https://github.com/FFmpeg/FFmpeg/blob/master/libavfilter/vf_deinterlace_vaapi.c>
>
> See function deint_vaapi_filter_frame.
>
>
> Oh thanks I would have never guessed that. It is indeed a bit
> counter-intuitive. And obviously the VA documentation does not tell you
> that.
> I was probably not able to see those flicker/judder by myself as on my
> computer
> I only have one backward frame. Anyway, thanks again.
> Also, LGTM.
>
>
> I may have noticed it more clearly because I had hacked in frame doubling
> too.
>
> Do you have any plans to add support for a frame rate doubling output?
> i.e. some sort of "Output one frame per field" option? Both dxva2 and
> vaapi deinterlacing lack this, and I think it's rather important.
> Watching any sort of interlaced sports content is way smoother.
>
> I would be happy to add it, it's quite straightforward, but I wasn't too
> sure how the option should be specified. Would each of the interlace
> modes "Bob" and "x" have an extra "Bobx2" and "xx2" counterpart (like the
> way Yadfifx2 works for software deinterlacing), or should it be a
> different option completely?
>
> Of course, you may already have this in your plans, in which case you may
> ignore me!
>
>
> Hello,
>
> We don't have that in our plans and we'll be happy to merge it.
> You can add "x2" or "bob2" options in the vaapi filters module. You'll
> be able to select it via the command line. If we add bob2 or x2 in the
> QT GUI, we need to have a working CPU implementation since we can't add
> a menu option that is working only for VAAPI. Sadly, we are not able
> (yet) to get HW module capabilities in order to fill the QT menu with
> adequate options. This won't be done for the 3.0 release (and probably
> not for the 4.0 one too).
>
> Regards,
> Thomas
>
>
> Thomas
>
> I am going to look at this soon, but I just wanted to check that you do want
> these as separate "2" variations as I noticed that Steve's recent commits
> have automatically enabled framerate doubling for both "Bob" and "x" (I
> haven't tested but the double frame rate flag seems to be set for
> both)....i.e. there aren't these separate "Bob2" and "x2" versions that you
> are proposing for vaapi.
>
>
> Actually in software "x" doesn't do the frame rate doubling and
> doesn't use frame history. But in Direct3D it does. The names outside
> of software may not match exactly what's available in the hardware.
> But all available modes should be presented and trying to match a
> corresponding software equivalent in terms of quality.
>
>
> Is that not down to whether you choose to output two frames or not though?
> i.e. you could just have easily as implemented DXVA2 X and Bob modes without
> doubling the frame rate? Or are you saying that DXVA2 expects to always
> output two frames for these deinterlace modes and it would be wrong to only
> output single frames (as you had it with your original DXVA2 deinterlace
> check-in).

No but it depends on the algorithm, it can switch between fields or
not (usually yes).

> Don't get me wrong - I personally much prefer it as you've done it, as I
> don't really see why you'd want to deinterlace 1080i input and *not* output
> double the frame rate. But there must be some use-cases I haven't thought
> of.

At least with CPU deinterlacing it's double the amount of CPU
processing. You may not always want that. And the default algorithms
(blend and x) don't double the framerate.

> But the same choice seems to exist for VAAPI - either loop through and
> output two frames each time, or don't. And the suggestion was that the
> existing "Bob" and "x" should continue to output single frames, whereas
> double frames should be reserved for new "Bob2" and "x2" variants....which
> would seem to be inconsistent vs DXVA2, although consistent with software.
>
> I presume it needs to be consistent across modules?

Only yadif2x suggests it doubles frames. For the others most users won't know.

>
> Regards.
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
>
>
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
>
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
>
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
>
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
>
>
>
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel