[vlc-devel] [PATCH] [RFC] decoder: add more decoder device types

Thomas Guillem thomas at gllm.fr
Mon Jun 24 10:32:27 CEST 2019



On Mon, Jun 24, 2019, at 10:26, Steve Lhomme wrote:
> On 2019-06-24 10:00, Thomas Guillem wrote:
> > 
> > On Mon, Jun 24, 2019, at 07:32, Steve Lhomme wrote:
> >> These are types needed to move the decoder pools that exist in various display
> >> modules closer to the decoder.
> >>
> >> That means we need to make the decision early on which flavor of OpenGL
> >> will be
> >> used before the OpenGL display module is created. Depending on the
> >> platform
> >> used it could be VAAPI, VDPAU, DXVA2, OpenGL PBO (persistent and
> >> non-persistent).
> >>
> >> And to do that we need to create the "decoder device" using the es_format_t
> >> that will be decoded (so we can pick a hardware decoder or a software one),
> >> which contains the codec, profile and dimensions. When any of these change we
> >> will need to create a new decoder device (or check the one we have is
> >> compatible).
> >> ---
> >>   include/vlc_codec.h | 9 +++++++++
> >>   1 file changed, 9 insertions(+)
> >>
> >> diff --git a/include/vlc_codec.h b/include/vlc_codec.h
> >> index 71f3e8439f..ec5d7e32af 100644
> >> --- a/include/vlc_codec.h
> >> +++ b/include/vlc_codec.h
> >> @@ -491,6 +491,9 @@ enum vlc_decoder_device_type
> >>       VLC_DECODER_DEVICE_DXVA2,
> >>       VLC_DECODER_DEVICE_D3D11VA,
> >>       VLC_DECODER_DEVICE_AWINDOW,
> >> +    VLC_DECODER_DEVICE_VULKAN,
> >> +    VLC_DECODER_DEVICE_PBO,
> > 
> > For software rendering, VK and GL can be merged. Both will need memory allocated with vlc_memfd(). That way, we could pass the fd to the VK/GL API.
> 
> Are you sure this can work when using Vulkan on Windows ?

I guess but I never tested it:

   VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT_KHR
   VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT_KHR

> > Decoders don't need to know about PBO. Indeed, PBO buffers (in non persistent) are just used as buffer to upload a picture without locking the whole GL context with glTexSubImage2D().
> 
> OK, so only 1 is necessary. This mode should already be updated to use 
> the "outside" CPU pool and upload the picture during Prepare.

I don't see why one is necessary. Just pass basic CPU pictures to OpenGL and Vulkan for now.
Then, GL could upload it using a PBO intermediate buffer. And vulkan could pass the FD directly (and fallback to memcpy it there is no FDs).

> 
> > And for persistent mode, I think we can drop it. Yes, from VLC point of view, there is one less memcpy, but I guess it is still done in the GL/Driver side since persistent buffers are mapped on the CPU.  Mapping buffers in GPU is worst by the way, since it is very slow to access them (for reference frames for example).
> 
> IMO in persistent mode the driver uses a backend copy. The visible on is 
> always the CPU one, but when it's needed in the GPU for rendering, it's 
> transfered there. At least on integrated GPUs it's the same memory so 
> it's not an issue, the driver doesn't need to do any trick and we get 
> good perf.
> 
> I don't feel comfortable dropping a zero copy mode unless we know it's 
> actually slower (no driver tricks).

That is what we decided with Niklas, RĂ©mi and Alexandre during the last vout workshop.

> 
> >> +    VLC_DECODER_DEVICE_MMAL,
> > 
> > Better to take care about MMAL later, when push model is merged and functional for most used HWs.
> 
> Yes, but it's better to keep it in mind and maybe keep compilation 
> working as we go forward.
> 
> >>   };
> >>   
> >>   /**
> >> @@ -514,6 +517,12 @@ typedef struct vlc_decoder_device
> >>        * The type of pointer will depend of the type:
> >>        * VAAPI: VADisplay
> >>        * VDPAU: vdp_t *
> >> +     * DXVA2: IDirect3DDevice9*
> >> +     * D3D11VA: ID3D11DeviceContext*
> >> +     * AWindow: android AWindowHandler*
> >> +     * Vulkan: vlc_vk_t*
> >> +     * OpenGL PBO: vlc_gl_t*
> >> +     * MMAL: MMAL_PORT_T*
> >>        */
> >>       void *opaque;
> >>   } vlc_decoder_device;
> >> -- 
> >> 2.17.1
> >>
> >> _______________________________________________
> >> vlc-devel mailing list
> >> To unsubscribe or modify your subscription options:
> >> https://mailman.videolan.org/listinfo/vlc-devel
> > _______________________________________________
> > vlc-devel mailing list
> > To unsubscribe or modify your subscription options:
> > https://mailman.videolan.org/listinfo/vlc-devel
> > 
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel


More information about the vlc-devel mailing list