[vlc-devel] Decoder/Display implementation selection

[Steve Lhomme]

On 2019-07-04 12:42, Jai Luthra wrote:
> On Thu, Jul 04, 2019 at 09:51:01AM +0200, Steve Lhomme wrote:
>> On 2019-07-04 9:23, Thomas Guillem wrote:
>>> If the dec-dev succeed to load one of these backend, it is very 
>>> likely that the hw decoder will succeed too. So we don't have your 
>>> problem on Linux.
>>
>> The HW may be loadable but the codec/profile not supported. For 
>> example as things are now NVDec only handles H264. So if you pick that 
>> automatically all the time, without knowing the codec, you'd never 
>> load the VDPAU dec-dev even though it can do HEVC (you can extend that 
>> to 10 bits, 4:4:4, etc instead of just the codec).
>>
>> We still need to be able to iterate between all the HW decoders before 
>> we pick one.
>>
>> NVDec being a decoder, that means once it's started, it might eat data 
>> before it selects a profile or realize it cannot handle it. We should 
>> fallback to lavc then. But from what I understand restarting the 
>> decoder will lose the bits it has already eaten ?
> 
> NVDec has a 'get device capabilities' API we can use to check if the 
> given codec+chroma+bitdepth is supported by the device.
> 
> If we probe bitdepth and chroma using the SPS in fmt_in.p_extra, then we 
> can call this api from the OpenDecoder callback itself, and fail there 
> if the profile isn't supported without eating up packets.
> 
> Although I'm not sure if p_extra is always populated, but usually demux 
> module does that?

You can check with TS for H264.

But in VP8/VP9 there's no p_extra in general. For VP9 which has a 
profile for 10 bits it's a problem. We have to probe the data to find 
which profile it's using.

For H264, HEVC, AV1 we probably have the demuxer/packetizer p_extra in 
the decoder. Maybe we need a VP9 packetizer and force using it when we 
don't have a p_extra.

>> (we could also support QSV decoders in the future which might support 
>> more codec/profiles than DXVA, so the issue is not just with NVDec)
> 
> I believe QSV can also tell if it can handle a profile or not via the 
> MFXVideoDECODE_Init call.

Yes, all hardware decoders should have this capability.

>>> If the user force one backend, it will try that backend and fallback 
>>> to SW if the backend can't be loaded.
>>
>> OK with that. There might still be a problem when falling back from 
>> NVDec/QSV to lavc.
>>
>>> On Windows, you have
>>> - D3D11
>>> - D3D9
>>> - NVDEC
>>>
>>> But you need to load lavc to know the correct backend, is that right 
>>> ? What I would do:
>>
>> It's the same as with VAAPI/VDPAU. lavc starts processing data and 
>> then ask for frames to decode to, with various VLD (including cuda, 
>> which we don't support for now). NVDec/QSV would be separate from that.
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel