[vlc-devel] [PATCH 05/14] filter: allow the owner not to provide a buffer callback
Alexandre Janniaux
ajanni at videolabs.io
Wed Sep 18 14:33:24 CEST 2019
Hi,
Just to have a common language:
+ a decoder device is a hint given by the windowing system to the decoder,
and potentially the decoder won't start without the correct decoder device
if it can have some meaning.
+ a video context is the common denominator between all the pictures going
out of a decoder, or a filter, although it could change in the middle of a
playback, and is pushed with the pictures (so it comes from before in the
pipeline, and can therefore be changed when creating new pictures).
Using the decoder device correctly would help zero copy in the pipeline,
if I understood correctly.
As I sent privately (and will publish soon), I've been writing an OpenGL
filter API for some time, which can be plugged to the classical filter_t
API.
When doing so, there is a filter_t creating an OpenGL context and the
OpenGL pipeline, and outputting native buffers. So it's built upon things
like GBM or Android Native Buffer. GBM is probably a good test case here:
The filter must be initialized with a gbm device, which represent the
device used for allocation. When supported by the EGL implementation, it
also represents the rendering device that will be used.
In this case, the most likely to happen is having a VAAPI decoder device
or NVDEC decoder device with X11 or Wayland window system and OpenGL video
output display. So we have vaapi/nvdec video context out of the decoder,
then our GBM opengl filter_t should produce a GBM video context, and the
last part, the OpenGL vout display, should be able to handle a GBM video
context.
Then, the question becomes: how to adapt the VAAPI decoder device in the
GBM OpenGL filter_t so as to use the same decoding and rendering device
everywhere?
Is it correct?
Regards,
--
Alexandre Janniaux
Videolabs
On Wed, Sep 18, 2019 at 02:32:01PM +0300, Rémi Denis-Courmont wrote:
> Hmm well yeah... in other words, it's not easy or at least not obvious in the general case.
>
> Also how does should an OpenGL shader-based filter if the decoder device is, say, NVdec, and the video context is X11? It knows neither of them. Sure it'd solvable, but to me it's not obvious at all, and I'd rather sort it out in a workshop than over a mailing list.
>
> Le 18 septembre 2019 12:15:30 GMT+03:00, Steve Lhomme <robux4 at ycbcr.xyz> a écrit :
> >On input of filters it's easy, they get a video context (or NULL for
> >CPU
> >based sources).
> >
> >The issue is when a filter needs to create a video context on its own.
> >For CPU to GPU filters it cannot use the input video context. Just like
> >
> >a decoder it should use the "decoder device" to create its output video
> >
> >context.
> >
> >So just like for the decoder, the owner of the filter provides a
> >"get_decoder_device" callback to get it.
> >
> >On 2019-09-18 10:39, Rémi Denis-Courmont wrote:
> >> Hi,
> >>
> >> I am not sure how hardware should be exposed to filters/converters. I
> >don't recall covering that part of the puzzle at the February workshop.
> >>
> >> Le 18 septembre 2019 09:00:41 GMT+03:00, Steve Lhomme
> ><robux4 at ycbcr.xyz> a écrit :
> >>> On 2019-09-17 17:32, Rémi Denis-Courmont wrote:
> >>>> Le tiistaina 17. syyskuuta 2019, 17.22.33 EEST Steve Lhomme a écrit
> >:
> >>>>> In this case we just allocate a picture from the filter output
> >>> format.
> >>>>> ---
> >>>>> include/vlc_filter.h | 7 ++++++-
> >>>>> 1 file changed, 6 insertions(+), 1 deletion(-)
> >>>>>
> >>>>> diff --git a/include/vlc_filter.h b/include/vlc_filter.h
> >>>>> index 0c756ce10ec..0bc10e91997 100644
> >>>>> --- a/include/vlc_filter.h
> >>>>> +++ b/include/vlc_filter.h
> >>>>> @@ -26,6 +26,7 @@
> >>>>> #define VLC_FILTER_H 1
> >>>>>
> >>>>> #include <vlc_es.h>
> >>>>> +#include <vlc_picture.h>
> >>>>>
> >>>>> /**
> >>>>> * \defgroup filter Filters
> >>>>> @@ -160,7 +161,11 @@ struct filter_t
> >>>>> */
> >>>>> static inline picture_t *filter_NewPicture( filter_t *p_filter
> >)
> >>>>> {
> >>>>> - picture_t *pic = p_filter->owner.video->buffer_new( p_filter
> >);
> >>>>> + picture_t *pic;
> >>>>> + if ( p_filter->owner.video != NULL &&
> >>> p_filter->owner.video->buffer_new
> >>>>> != NULL)
> >>>>
> >>>> Two checks look redundant, though anyway this callback should go
> >away
> >>>> altogether eventually.
> >>>
> >>> Yes, it will go away.
> >>>
> >>> The owner will have the "get_device" callback so it's possible at
> >some
> >>> point in the code to have one callback and not the other. But in the
> >>> end
> >>> this code won't be there anymore.
> >>>
> >>>>> + pic = p_filter->owner.video->buffer_new( p_filter );
> >>>>> + else
> >>>>> + pic = picture_NewFromFormat( &p_filter->fmt_out.video );
> >>>>> if( pic == NULL )
> >>>>> msg_Warn( p_filter, "can't get output picture" );
> >>>>> return pic;
> >>>>
> >>>>
> >>>> --
> >>>> Rémi Denis-Courmont
> >>>> http://www.remlab.net/
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> vlc-devel mailing list
> >>>> To unsubscribe or modify your subscription options:
> >>>> https://mailman.videolan.org/listinfo/vlc-devel
> >>>>
> >>> _______________________________________________
> >>> vlc-devel mailing list
> >>> To unsubscribe or modify your subscription options:
> >>> https://mailman.videolan.org/listinfo/vlc-devel
> >>
> >> --
> >> Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez
> >excuser ma brièveté.
> >>
> >>
> >> _______________________________________________
> >> vlc-devel mailing list
> >> To unsubscribe or modify your subscription options:
> >> https://mailman.videolan.org/listinfo/vlc-devel
> >>
> >_______________________________________________
> >vlc-devel mailing list
> >To unsubscribe or modify your subscription options:
> >https://mailman.videolan.org/listinfo/vlc-devel
>
> --
> Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma brièveté.
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
More information about the vlc-devel
mailing list