[vlc-devel] [PATCH] [RFC] decoder: add more decoder device types

[Steve Lhomme]

On 2019-06-24 8:47, Steve Lhomme wrote:
> On 2019-06-24 7:31, Steve Lhomme wrote:
>> These are types needed to move the decoder pools that exist in various 
>> display
>> modules closer to the decoder.
>>
>> That means we need to make the decision early on which flavor of 
>> OpenGL will be
>> used before the OpenGL display module is created. Depending on the 
>> platform
>> used it could be VAAPI, VDPAU, DXVA2, OpenGL PBO (persistent and 
>> non-persistent).
>>
>> And to do that we need to create the "decoder device" using the 
>> es_format_t
>> that will be decoded (so we can pick a hardware decoder or a software 
>> one),
>> which contains the codec, profile and dimensions. When any of these 
>> change we
>> will need to create a new decoder device (or check the one we have is
>> compatible).
> 
> That would be the fmt_in of the decoder, whearas right now we only get 
> the fmt_out.video in the display module. This is quite a stretch from 
> what we have now.
> 
> In the case of the hardware decoder pick we don't really need it. In 
> automatic mode we can already pick the best candidate. If it doesn't 
> work we fallback to software decoding. On Windows it's not an issue, we 
> don't do a zero copy rendering in that case.
> 
> In OpenGL (not automatically select on Windows) it's trickier as the 
> fallback to software decoding can still use zero copy as done in 
> converter_sw.c. The pool that is created there should be moved to the 
> decoder. To do that we need the same vlc_gl_t that is used by the 
> display module. But if DXVA2 (or VAAPI or VDPAU on other platforms) was 
> "best picked" we don't have this information.
> 
> We need a way to retry to create the decoder device (and display module) 
> if the selected decoder device didn't work.

Given the hardware decoder pick (with lavc) is (and will still be) 
driven by the pixelformat given to the UpdateFormat, the retry can be 
handled that way. When we're set on decoding in software, the decoder 
device (and video context and pool) is set to use software. We will 
still need to know if it's OpenGL rendering or not.

>> ---
>>   include/vlc_codec.h | 9 +++++++++
>>   1 file changed, 9 insertions(+)
>>
>> diff --git a/include/vlc_codec.h b/include/vlc_codec.h
>> index 71f3e8439f..ec5d7e32af 100644
>> --- a/include/vlc_codec.h
>> +++ b/include/vlc_codec.h
>> @@ -491,6 +491,9 @@ enum vlc_decoder_device_type
>>       VLC_DECODER_DEVICE_DXVA2,
>>       VLC_DECODER_DEVICE_D3D11VA,
>>       VLC_DECODER_DEVICE_AWINDOW,
>> +    VLC_DECODER_DEVICE_VULKAN,
>> +    VLC_DECODER_DEVICE_PBO,
>> +    VLC_DECODER_DEVICE_MMAL,
>>   };
>>   /**
>> @@ -514,6 +517,12 @@ typedef struct vlc_decoder_device
>>        * The type of pointer will depend of the type:
>>        * VAAPI: VADisplay
>>        * VDPAU: vdp_t *
>> +     * DXVA2: IDirect3DDevice9*
>> +     * D3D11VA: ID3D11DeviceContext*
>> +     * AWindow: android AWindowHandler*
>> +     * Vulkan: vlc_vk_t*
>> +     * OpenGL PBO: vlc_gl_t*
>> +     * MMAL: MMAL_PORT_T*
>>        */
>>       void *opaque;
>>   } vlc_decoder_device;
>> -- 
>> 2.17.1
>>
>> _______________________________________________
>> vlc-devel mailing list
>> To unsubscribe or modify your subscription options:
>> https://mailman.videolan.org/listinfo/vlc-devel
>>
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel