[vlc-devel] DXVA no CPU access design

Rémi Denis-Courmont remi at remlab.net
Tue Apr 21 20:00:07 CEST 2015

Le 2015-04-21 19:24, Steve Lhomme a écrit :
> It seems that we're turning around discussing solutions that never
> work. So let's summarize the constraints we have to try to find a
> working solution. Unless we code a DXVA2 decoder from scratch (that
> may be a possibility since we have packtizers), we have to deal with
> how avcodec works
> 1/ the avcodec decoder is opened, found to be capable of handling the
> codec, and so is used.
> 2/ It then starts decoding the input stream.
> 3/ When it has enough data to write to an output frame, it first 
> tries
> to define the output format using the decoder->get_format() calback,
> by providing some possible ones, including the hardware accelerated
> variants. This is where we open our DXVA2 module.

The libavcodec plugin should update ->fmt_out then call 
decoder_UpdateVideoFormat(). Currently this is postponed to 
ffmpeg_NewPictBuf(). IMU, that's what libavcodec get_format() is meant 
for. At that point, the vout (or more generally, the decoder output) 
will be initialized. Error checking is also possible.

I am not sure that if all decoder owners implement 
->pf_vout_format_update() correctly yet, so that might need to be fixed.

> 4/ If we finds a va decoder, we set it up using vlc_va_Setup()

Again, vlc_va_Setup() is only a work around for limitations of 
libavcodec. I am not sure the problem even still exists. Hardware 
initialization should be in vlc_va_New().

> At this point the decoder is provided with a `hwaccel_context` that
> contains the list of `surface` it can use to read the decoded frames
> coming out of the va.
> 5/ The decoder decodes the input stream by using the provided 
> hardware
> accelerated surfaces provided in the Setup, via the vlc_va_Get() 
> which
> picks which surface is free to use. It gets a surface and look for 
> the
> index in its internal buffer in ff_dxva2_get_surface_index()
> http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavcodec/dxva2.c;h=c1c7681402ab6615252e8e7205c1d19ea9e055d0;hb=HEAD#l38

In principles, the decoder should always call ffmpeg_NewPictBuf() in 
->get_buffer(). Currently, it only does so in case of DR.

> 6/ an AVFrame eventually comes out of the video decoder
> The AVFrame planes/surfaces is turned into a picture_t:
> 7/ decoder_NewPicture() is called with the decoder_t
> 8/ which then calls decoder_UpdateVideoFormat() which creates the 
> vout
> if there's not one already usable.
> 9/ the vout is created in a different thread and the decoder thread
> 10/ the VOUT_CONTROL_INIT control signal is pushed and then the
> decoder thread wait for the control message to be finished handling,
> at which point the vout is created (or won't be). The vout is created
> though a vout_configuration_t structure.
> 11/ a picture_t is created in decoder_NewPicture().
> Because of #5 we cannot go easily around the design of avcodec. It
> needs to know the table of surfaces that will be used by the decoded.
> The vout is only involved after the first AVFrame with a DXVA surface
> comes out of the decoder.
> For these surfaces to be used to render, the D3D9 output needs to use
> the D3D9 handle and the same D3D9 device as the DXVA decoder. So at
> least these 2 objects need to be passed along the way. They are D3D
> objects so the receiving vout can add/release references to these
> objects so there won't be dereferencing issues of these pointers
> during the lifetime of the module(s).
> The main problem we have is how to share these objects in a structure
> that is safe from dereferencing between #8 and #11 (and also when the
> vout is reinited). IMO vout_configuration_t and the pointers inside 
> is
> safe during the duration of the vout create/reinit call. The vout
> should not keep a pointer passed through there though.

First I disagree with the notion that it is safe, because of threading, 
comparing and copying.

Making the whole thing safe is a futile exercise, because the video 
output decides where the video is rendered. The decoder cannot tell the 
video output, at least not without breaking the existing GUI and Libvlc.

Rémi Denis-Courmont

More information about the vlc-devel mailing list