[vlc-devel] [PATCH] Enable DXVA2 deinterlacing frame doubling
Oliver Collyer
ovcollyer at mac.com
Thu Jun 22 08:59:49 CEST 2017
Hi Steve
Yes I had originally posted the file in the forums but decided to make a patch for it.
You are right there are more enhancements to be made; I could see that it was only looking for the simplest deinterlacing mode that didn't require forward or backwards references, and it seemed an obvious first step improvement to at least make it double the frames so that 1080i50/60 sources look smooth. As it stands, it's basically useless. I'm using libVLC in my own project which uses interlaced sources so it was beneficial to get this going and I wanted to share.
I noticed many changes recently to the way the surfaces are stored in DXVA2 so I suspect this patch may need to be modified to grab the decoded surface from the context instead of the picsys. I was going to look at doing this but put it on hold when I discovered the Windows build has been completely broken by the aligned_alloc changes.
Anyway do what you want with the patch...or not!
Regards
Oliver
> 22 Jun 2017, at 09:44, Steve Lhomme <robux4 at gmail.com> wrote:
>
> Hi,
>
> I have seen this code here:
> https://forum.videolan.org/viewtopic.php?p=456307#p456333
>
> It's doubling as in copying the frames no matter what. If you look at
> the software deinterlacer (on which this patch seems to be based on),
> the frame doubling depends on the algorithm in use. But right now we
> select a video processor that has NumForwardRefSamples and
> NumBackwardRefSamples to 0. So what we have right now is the
> equivalent of "blend" that doesn't double the frames. So I would say
> as of now it is not correct.
>
> I am working on cleaning this patch (but got side tracked in the
> meantime) and supporting more algorithm found here :
> https://msdn.microsoft.com/en-us/library/windows/desktop/ms703795(v=vs.85).aspx
>
> I also have some cleaning and code sharing in line for the software
> deinterlacer.
>
>> On Wed, Jun 21, 2017 at 5:44 PM, Oliver Collyer <ovcollyer at mac.com> wrote:
>> (Sorry I don't think it worked with my attachment, inlining the patch this time)
>>
>> The current DXVA2 deinterlacing module lacked the frame-doubling that gives interlaced sources proper smooth output, this patch addresses this.
>>
>> It was also hardcoded to assume 1080i60 input frame rate, this version now correctly deduces the frame rate, taking inspiration from similar code in the software deinterlace.c.
>> ---
>> modules/video_output/win32/dxva2_deinterlace.c | 107 ++++++++++++++++++-------
>> 1 file changed, 77 insertions(+), 30 deletions(-)
>>
>> diff --git a/modules/video_output/win32/dxva2_deinterlace.c b/modules/video_output/win32/dxva2_deinterlace.c
>> index b01e46b282..4aeaafe8a2 100644
>> --- a/modules/video_output/win32/dxva2_deinterlace.c
>> +++ b/modules/video_output/win32/dxva2_deinterlace.c
>> @@ -38,6 +38,13 @@
>> #include <dxva2api.h>
>> #include "../../video_chroma/d3d9_fmt.h"
>>
>> +#define METADATA_SIZE 3
>> +
>> +typedef struct {
>> + mtime_t pi_date[METADATA_SIZE];
>> + int pi_nb_fields[METADATA_SIZE];
>> +} metadata_history_t;
>> +
>> struct filter_sys_t
>> {
>> HINSTANCE hdecoder_dll;
>> @@ -46,6 +53,7 @@ struct filter_sys_t
>> IDirect3DDevice9 *d3ddev;
>> IDirectXVideoProcessor *processor;
>> IDirect3DSurface9 *hw_surface;
>> + metadata_history_t meta;
>> };
>>
>> static picture_t *Deinterlace(filter_t *filter, picture_t *src)
>> @@ -56,20 +64,35 @@ static picture_t *Deinterlace(filter_t *filter, picture_t *src)
>> DXVA2_VideoSample sample = {0};
>> D3DSURFACE_DESC dstDesc;
>>
>> + for (int i = 1; i < METADATA_SIZE; i ++) {
>> + sys->meta.pi_date[i-1] = sys->meta.pi_date[i];
>> + sys->meta.pi_nb_fields[i-1] = sys->meta.pi_nb_fields[i];
>> + }
>> + sys->meta.pi_date[METADATA_SIZE-1] = src->date;
>> + sys->meta.pi_nb_fields[METADATA_SIZE-1] = src->i_nb_fields;
>> +
>> + mtime_t i_field_dur = 0;
>> + int i = 0;
>> + for ( ; i < METADATA_SIZE-1; i ++)
>> + if (sys->meta.pi_date[i] > VLC_TS_INVALID)
>> + break;
>> + if (i < METADATA_SIZE-1) {
>> + int i_fields_total = 0;
>> + for (int j = i; j < METADATA_SIZE-1; j ++)
>> + i_fields_total += sys->meta.pi_nb_fields[j];
>> + i_field_dur = (src->date - sys->meta.pi_date[i]) / i_fields_total;
>> + }
>> +
>> hr = IDirect3DSurface9_GetDesc( src->p_sys->surface, &dstDesc );
>> if (FAILED(hr))
>> return src; /* cannot deinterlace without copying fields */
>>
>> - picture_t *dst = filter_NewPicture(filter);
>> - if (dst == NULL)
>> - return src; /* cannot deinterlace without copying fields */
>> -
>> sample.SrcSurface = src->p_sys->surface;
>> sample.SampleFormat.SampleFormat = src->b_top_field_first ?
>> DXVA2_SampleFieldInterleavedEvenFirst :
>> DXVA2_SampleFieldInterleavedOddFirst;
>> sample.Start = 0;
>> - sample.End = sample.Start + 333666;
>> + sample.End = sample.Start + i_field_dur * 20;
>> sample.SampleData = DXVA2_SampleData_RFF_TFF_Present;
>> if (src->b_top_field_first)
>> sample.SampleData |= DXVA2_SampleData_TFF;
>> @@ -78,35 +101,53 @@ static picture_t *Deinterlace(filter_t *filter, picture_t *src)
>> sample.DstRect = sample.SrcRect;
>> sample.PlanarAlpha = DXVA2_Fixed32OpaqueAlpha();
>>
>> - params.TargetFrame = sample.Start;
>> - params.TargetRect = sample.DstRect;
>> - params.DestData = sample.SampleData;
>> - params.Alpha = DXVA2_Fixed32OpaqueAlpha();
>> - params.DestFormat.SampleFormat = DXVA2_SampleProgressiveFrame;
>> - params.BackgroundColor.Alpha = 0xFFFF;
>> -
>> - hr = IDirectXVideoProcessor_VideoProcessBlt( sys->processor,
>> - sys->hw_surface,
>> - ¶ms,
>> - &sample,
>> - 1, NULL );
>> - if (FAILED(hr))
>> - goto error;
>> + picture_t *dst[2] = {NULL, NULL};
>> + for (int i = 0; i < 2; i ++) {
>> + dst[i] = filter_NewPicture(filter);
>> + if (dst[i] == NULL)
>> + goto error;
>> + if (i)
>> + dst[i-1]->p_next = dst[i];
>> +
>> + params.TargetFrame = sample.Start + (sample.End - sample.Start) * i / 2;
>> + params.TargetRect = sample.DstRect;
>> + params.DestData = sample.SampleData;
>> + params.Alpha = DXVA2_Fixed32OpaqueAlpha();
>> + params.DestFormat.SampleFormat = DXVA2_SampleProgressiveFrame;
>> + params.BackgroundColor.Alpha = 0xFFFF;
>> +
>> + hr = IDirectXVideoProcessor_VideoProcessBlt( sys->processor,
>> + sys->hw_surface,
>> + ¶ms,
>> + &sample,
>> + 1, NULL );
>> + if (FAILED(hr))
>> + goto error;
>> +
>> + hr = IDirect3DDevice9_StretchRect( sys->d3ddev,
>> + sys->hw_surface, NULL,
>> + dst[i]->p_sys->surface, NULL,
>> + D3DTEXF_NONE);
>> + if (FAILED(hr))
>> + goto error;
>> +
>> + picture_CopyProperties(dst[i], src);
>> + dst[i]->b_progressive = true;
>> + dst[i]->i_nb_fields = 1;
>> + }
>>
>> - hr = IDirect3DDevice9_StretchRect( sys->d3ddev,
>> - sys->hw_surface, NULL,
>> - dst->p_sys->surface, NULL,
>> - D3DTEXF_NONE);
>> - if (FAILED(hr))
>> - goto error;
>> + dst[0]->date = src->date;
>> + if (dst[0]->date > VLC_TS_INVALID)
>> + dst[1]->date = dst[0]->date + i_field_dur;
>> + else
>> + dst[1]->date = VLC_TS_INVALID;
>>
>> - picture_CopyProperties(dst, src);
>> picture_Release(src);
>> - dst->b_progressive = true;
>> - dst->i_nb_fields = 1;
>> - return dst;
>> + return dst[0];
>> error:
>> - picture_Release(dst);
>> + for (int i = 0; i < 2; i ++)
>> + if (dst[i])
>> + picture_Release(dst[i]);
>> return src;
>> }
>>
>> @@ -239,6 +280,12 @@ static int Open(vlc_object_t *obj)
>>
>> sys->hdecoder_dll = hdecoder_dll;
>> sys->d3d9_dll = d3d9_dll;
>> +
>> + for (int i = 0; i < METADATA_SIZE; i ++) {
>> + sys->meta.pi_date[i] = VLC_TS_INVALID;
>> + sys->meta.pi_nb_fields[i] = 2;
>> + }
>> +
>> filter->pf_video_filter = Deinterlace;
>> filter->p_sys = sys;
>>
>> --
>> 2.11.0
>>
>> _______________________________________________
>> vlc-devel mailing list
>> To unsubscribe or modify your subscription options:
>> https://mailman.videolan.org/listinfo/vlc-devel
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
More information about the vlc-devel
mailing list