[vlc-devel] [PATCH] vout: create/release the decoder device when the window is enabled/disabled

[Rémi Denis-Courmont]

Hi,

The contemporary reality is such that the decoding DSP is a dedicated piece of fixed function hardware, either with a dedicated API or a dedicated component within a more general API (e.g. NVDEC within CUDA). Filtering and the rest is done by the (GP)GPU with programmable hardware and possibly completely separate API than decoding, e.g VK or GL.

I don't see how it can even work if video context and decoder device are the same thing. You literally can't do filtering and rendering with a decoder. There may be interdependence to take into account when creating those objects, but that does not mean they're the same thing.

And that's exactly why I don't think it's as simple and obvious as exposing the decoder to filters.

Le 19 septembre 2019 15:18:12 GMT+03:00, Alexandre Janniaux <ajanni at videolabs.io> a écrit :
>Hi,
>
>Wasn't the workshop only about window / decoder, and video context the
>previous name of what become decoder_device ?
>
>It might become confusion with Steve's video context(s) definition.
>
>What would create the (only?) video context and what do you want the
>video
>context to bring to the filter pipeline in a real world case ?
>
>Regards,
>--
>Alexandre Janniaux
>Videolabs
>
>On Thu, Sep 19, 2019 at 03:06:20PM +0300, Rémi Denis-Courmont wrote:
>> No. The name is correct. That's thewhole point of distinguish between
>decoder device and video context.
>>
>> The decoder device abstracts the video decoding DSP, and shields the
>decoder abstraction from depending on windowing systems (so that
>headless decoding is possible).
>>
>> The video context provides abstracts the GPU for video processing
>purpose - filter, conversion, blend and render. The video context is
>guaranteed to have a window underneath it, since rendering may depend
>on the intended display controller.
>>
>> Le 19 septembre 2019 10:19:13 GMT+03:00, Steve Lhomme
><robux4 at ycbcr.xyz> a écrit :
>> >On 2019-09-19 8:40, Rémi Denis-Courmont wrote:
>> >> It's called "decoder device" because it's for the decoder.
>> >
>> >The name itself is not correct. As explained, it is also used by
>> >filters
>> >to create video context from scratch (that will output pictures
>> >compatible with the "decoder device" used by the display module). We
>> >could call it "output device" or even "output adapter" since it
>seems
>> >to
>> >always be tied to a display adapter.
>> >
>> >In the case of transcoding it's actually the device used by the
>> >encoder,
>> >that the decoder needs to try to match.
>> >
>> >As Alexandre reminded, it's mostly a hint for the decoder (and
>filters)
>> >
>> >to match the output. So we could also call it "device hint".
>> >
>> >> And the decoder device does not have to come from a window. That's
>> >not how it's going to work for transcoding.
>> >
>> >It doesn't have to in most cases. But for devices like VAAPI it
>needs
>> >to
>> >know the type and ID of the window so the decoder can actually be
>used
>> >(not sure if there are encoders for VAAPI and if they can use a
>dummy
>> >VADisplay value). See the Open() of the VAAPI decoder device [1]. No
>> >window => no "decoder device". No "decoder device" => no decoder.
>> >
>> >We actually designed the "decoder device" during the workshop
>because
>> >of
>> >that. I did not originally go that way since on Windows I never need
>to
>> >
>> >know the window (and in fact I can't know the adapter from the
>HWND).
>> >
>> >[1]
>>
>>https://code.videolan.org/videolan/vlc/blob/master/modules/hw/vaapi/decoder_device.c#L223
>> >
>> >> Le 19 septembre 2019 09:30:30 GMT+03:00, Steve Lhomme
>> ><robux4 at ycbcr.xyz> a écrit :
>> >>> On 2019-09-19 8:11, Steve Lhomme wrote:
>> >>>> On 2019-09-18 18:16, Rémi Denis-Courmont wrote:
>> >>>>> Le keskiviikkona 18. syyskuuta 2019, 17.13.18 EEST Steve Lhomme
>a
>> >>> écrit :
>> >>>>>> The decoder device ("dec-dev") should be tied to the lifecyle
>of
>> >>> the
>> >>>>>> window.
>> >>>>>
>> >>>>> Well, no. It's tied to an enabled window, which is longer than
>a
>> >>> vout,
>> >>>>> but
>> >>>>> shorter than a window.
>> >>>>
>> >>>> I can change it to:
>> >>>> The decoder device ("dec-dev") should be tied to the lifecyle of
>an
>> >>>> enabled window.
>> >>>>
>> >>>>> And AFAIK, the vout should not know about the decoder or
>decoder
>> >>>>> device at
>> >>>>> all. It should use the video context only.
>> >>>>
>> >>>> And how do you plan to convey this decoder device instance to
>the
>> >>>> decoder/filters without going through the vout ?
>> >>>
>> >>> For the record, the decoder device is created with a
>vout_window_t,
>> >>> because in some cases (VAAPI for example) it needs to know about
>> >which
>> >>> window is used. This vout_window_t is created and stored by the
>> >vout.
>> >>> _______________________________________________
>> >>> vlc-devel mailing list
>> >>> To unsubscribe or modify your subscription options:
>> >>> https://mailman.videolan.org/listinfo/vlc-devel
>> >>
>> >> --
>> >> Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez
>> >excuser ma brièveté.
>> >>
>> >>
>> >> _______________________________________________
>> >> vlc-devel mailing list
>> >> To unsubscribe or modify your subscription options:
>> >> https://mailman.videolan.org/listinfo/vlc-devel
>> >>
>> >_______________________________________________
>> >vlc-devel mailing list
>> >To unsubscribe or modify your subscription options:
>> >https://mailman.videolan.org/listinfo/vlc-devel
>>
>> --
>> Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez
>excuser ma brièveté.
>
>> _______________________________________________
>> vlc-devel mailing list
>> To unsubscribe or modify your subscription options:
>> https://mailman.videolan.org/listinfo/vlc-devel
>_______________________________________________
>vlc-devel mailing list
>To unsubscribe or modify your subscription options:
>https://mailman.videolan.org/listinfo/vlc-devel

-- 
Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma brièveté.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20190919/c312564a/attachment-0001.html>