[vlc-devel] [PATCH] vout: create/release the decoder device when the window is enabled/disabled

Thu Sep 19 14:36:57 CEST 2019

On 2019-09-19 14:28, Rémi Denis-Courmont wrote:
> Hi,
> 
> The contemporary reality is such that the decoding DSP is a dedicated piece of fixed function hardware, either with a dedicated API or a dedicated component within a more general API (e.g. NVDEC within CUDA). Filtering and the rest is done by the (GP)GPU with programmable hardware and possibly completely separate API than decoding, e.g VK or GL.
> 
> I don't see how it can even work if video context and decoder device are the same thing. You literally can't do filtering and rendering with a decoder. There may be interdependence to take into account when creating those objects, but that does not mean they're the same thing.
> 
> And that's exactly why I don't think it's as simple and obvious as exposing the decoder to filters.

I don't think any one proposed that.
In the 3 video contexts I implemented (D3D9, D3D11, NVDEC) the video 
context only has extra info about the surface characteristics. And info 
about the adapter/instance used so the receiver can use the same. This 
is also information found in the decoder device.

As said in another post a decoder/filter could push a video context not 
matching the "pipeline decoder device/adapter hint". An adapter will be 
used to get back to what the pipeline wants.

> Le 19 septembre 2019 15:18:12 GMT+03:00, Alexandre Janniaux <ajanni at videolabs.io> a écrit :
>> Hi,
>>
>> Wasn't the workshop only about window / decoder, and video context the
>> previous name of what become decoder_device ?
>>
>> It might become confusion with Steve's video context(s) definition.
>>
>> What would create the (only?) video context and what do you want the
>> video
>> context to bring to the filter pipeline in a real world case ?
>>
>> Regards,
>> --
>> Alexandre Janniaux
>> Videolabs
>>
>> On Thu, Sep 19, 2019 at 03:06:20PM +0300, Rémi Denis-Courmont wrote:
>>> No. The name is correct. That's thewhole point of distinguish between
>> decoder device and video context.
>>>
>>> The decoder device abstracts the video decoding DSP, and shields the
>> decoder abstraction from depending on windowing systems (so that
>> headless decoding is possible).
>>>
>>> The video context provides abstracts the GPU for video processing
>> purpose - filter, conversion, blend and render. The video context is
>> guaranteed to have a window underneath it, since rendering may depend
>> on the intended display controller.
>>>
>>> Le 19 septembre 2019 10:19:13 GMT+03:00, Steve Lhomme
>> <robux4 at ycbcr.xyz> a écrit :
>>>> On 2019-09-19 8:40, Rémi Denis-Courmont wrote:
>>>>> It's called "decoder device" because it's for the decoder.
>>>>
>>>> The name itself is not correct. As explained, it is also used by
>>>> filters
>>>> to create video context from scratch (that will output pictures
>>>> compatible with the "decoder device" used by the display module). We
>>>> could call it "output device" or even "output adapter" since it
>> seems
>>>> to
>>>> always be tied to a display adapter.
>>>>
>>>> In the case of transcoding it's actually the device used by the
>>>> encoder,
>>>> that the decoder needs to try to match.
>>>>
>>>> As Alexandre reminded, it's mostly a hint for the decoder (and
>> filters)
>>>>
>>>> to match the output. So we could also call it "device hint".
>>>>
>>>>> And the decoder device does not have to come from a window. That's
>>>> not how it's going to work for transcoding.
>>>>
>>>> It doesn't have to in most cases. But for devices like VAAPI it
>> needs
>>>> to
>>>> know the type and ID of the window so the decoder can actually be
>> used
>>>> (not sure if there are encoders for VAAPI and if they can use a
>> dummy
>>>> VADisplay value). See the Open() of the VAAPI decoder device [1]. No
>>>> window => no "decoder device". No "decoder device" => no decoder.
>>>>
>>>> We actually designed the "decoder device" during the workshop
>> because
>>>> of
>>>> that. I did not originally go that way since on Windows I never need
>> to
>>>>
>>>> know the window (and in fact I can't know the adapter from the
>> HWND).
>>>>
>>>> [1]
>>>
>>> https://code.videolan.org/videolan/vlc/blob/master/modules/hw/vaapi/decoder_device.c#L223
>>>>
>>>>> Le 19 septembre 2019 09:30:30 GMT+03:00, Steve Lhomme
>>>> <robux4 at ycbcr.xyz> a écrit :
>>>>>> On 2019-09-19 8:11, Steve Lhomme wrote:
>>>>>>> On 2019-09-18 18:16, Rémi Denis-Courmont wrote:
>>>>>>>> Le keskiviikkona 18. syyskuuta 2019, 17.13.18 EEST Steve Lhomme
>> a
>>>>>> écrit :
>>>>>>>>> The decoder device ("dec-dev") should be tied to the lifecyle
>> of
>>>>>> the
>>>>>>>>> window.
>>>>>>>>
>>>>>>>> Well, no. It's tied to an enabled window, which is longer than
>> a
>>>>>> vout,
>>>>>>>> but
>>>>>>>> shorter than a window.
>>>>>>>
>>>>>>> I can change it to:
>>>>>>> The decoder device ("dec-dev") should be tied to the lifecyle of
>> an
>>>>>>> enabled window.
>>>>>>>
>>>>>>>> And AFAIK, the vout should not know about the decoder or
>> decoder
>>>>>>>> device at
>>>>>>>> all. It should use the video context only.
>>>>>>>
>>>>>>> And how do you plan to convey this decoder device instance to
>> the
>>>>>>> decoder/filters without going through the vout ?
>>>>>>
>>>>>> For the record, the decoder device is created with a
>> vout_window_t,
>>>>>> because in some cases (VAAPI for example) it needs to know about
>>>> which
>>>>>> window is used. This vout_window_t is created and stored by the
>>>> vout.
>>>>>> _______________________________________________
>>>>>> vlc-devel mailing list
>>>>>> To unsubscribe or modify your subscription options:
>>>>>> https://mailman.videolan.org/listinfo/vlc-devel
>>>>>
>>>>> --
>>>>> Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez
>>>> excuser ma brièveté.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> vlc-devel mailing list
>>>>> To unsubscribe or modify your subscription options:
>>>>> https://mailman.videolan.org/listinfo/vlc-devel
>>>>>
>>>> _______________________________________________
>>>> vlc-devel mailing list
>>>> To unsubscribe or modify your subscription options:
>>>> https://mailman.videolan.org/listinfo/vlc-devel
>>>
>>> --
>>> Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez
>> excuser ma brièveté.
>>
>>> _______________________________________________
>>> vlc-devel mailing list
>>> To unsubscribe or modify your subscription options:
>>> https://mailman.videolan.org/listinfo/vlc-devel
>> _______________________________________________
>> vlc-devel mailing list
>> To unsubscribe or modify your subscription options:
>> https://mailman.videolan.org/listinfo/vlc-devel
> 
> -- 
> Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma brièveté.
> 
> 
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
> 