[vlc-devel] DXVA no CPU access design

[Steve Lhomme]

On Tue, Apr 21, 2015 at 8:46 PM, Julian Scheel <julian at jusst.de> wrote:
> Am 21.04.15 um 20:00 schrieb Rémi Denis-Courmont:
>>
>> Le 2015-04-21 19:24, Steve Lhomme a écrit :
>>>
>>> It seems that we're turning around discussing solutions that never
>>> work. So let's summarize the constraints we have to try to find a
>>> working solution. Unless we code a DXVA2 decoder from scratch (that
>>> may be a possibility since we have packtizers), we have to deal with
>>> how avcodec works
>>>
>>> 1/ the avcodec decoder is opened, found to be capable of handling the
>>> codec, and so is used.
>>>
>>> 2/ It then starts decoding the input stream.
>>>
>>> 3/ When it has enough data to write to an output frame, it first tries
>>> to define the output format using the decoder->get_format() calback,
>>> by providing some possible ones, including the hardware accelerated
>>> variants. This is where we open our DXVA2 module.
>>
>>
>> The libavcodec plugin should update ->fmt_out then call
>> decoder_UpdateVideoFormat(). Currently this is postponed to
>> ffmpeg_NewPictBuf(). IMU, that's what libavcodec get_format() is meant
>> for. At that point, the vout (or more generally, the decoder output)
>> will be initialized. Error checking is also possible.
>
>
> Afaicr decoder_UpdateVideoFormat() is not required to be called explicitly
> by the codec. It's sufficient to just update the fmt_out of the codec and
> the next decoder_PictureNew() will call decoder_UpdateVideoFormat() so that
> the new format is populated down the vout chain.
>
>> I am not sure that if all decoder owners implement
>> ->pf_vout_format_update() correctly yet, so that might need to be fixed.
>>
>>> 4/ If we finds a va decoder, we set it up using vlc_va_Setup()
>>
>>
>> Again, vlc_va_Setup() is only a work around for limitations of
>> libavcodec. I am not sure the problem even still exists. Hardware
>> initialization should be in vlc_va_New().
>>
>>> At this point the decoder is provided with a `hwaccel_context` that
>>> contains the list of `surface` it can use to read the decoded frames
>>> coming out of the va.
>
>
> Where is this hwaccel_context coming from?

It's a structure inside the vlc_va_sys_t, we pass a pointer to this
internal structure. The values are used right away to fill the avcodec
va structure.

> And what prevents you from retrieving it in the vout and putting it into a picture_sys_t?

Because avcodec will have used the provided buffers before the vout is
created. See the flow I described above. It seems that the changes
made by Rémi last night changed the dynamic. The vout is now created
before avcodec tries to get an AVFrame to write to. So it might be
possible to do the delayed init.

One possible downside is that if the HW init fails for some reason,
the playback won't work at all.