[vlc-devel] Direct3D9 + DXVA direct rendering

[Steve Lhomme]

On Mon, Apr 13, 2015 at 6:51 PM, Rémi Denis-Courmont <remi at remlab.net> wrote:
> Le lundi 13 avril 2015, 17:17:57 Steve Lhomme a écrit :
>> While this is primarily working, when closing VLC I get all sorts of
>> crashes with threads using dead pointers. For example the surfaces
>> pushed in the display FIFO belong to the decoder. So if the decoder is
>> closed, there are some picture_t in the FIFO pointing to invalid data.
>
> Yes. There is currently no way to allocate picture from the decoder. And even
> for hardware surfaces, I would expect the video output to keep responsibility
> for determining the output target device and thus for allocating surface
> handles.

In the case of DXVA, it's tied to D3D9 and the formats it outputs are
meant to be used as D3D9 surfaces directly without conversion. So
there's no much to worry there.

>> If they belonged to the vout then the decoder wouldn't work if the
>> vout not available or reset for some reason.
>
> The picture pool does not change if the decoder output format does not change.
> Otherwise, there would be no ways for decoders to retain reference frames.

Good to know. I need to keep track of when that display FIFO is flushed.

> Also in principles, a picture remains valid as long as it is referenced by the
> decoder. If the format changed after the picture was allocated, the picture
> can no longer be rendered, but the picture structure is still valid.

The surfaces passed to the vout are not the ones used internally by
DXVA. I do some surface copying but inside the GPU.

>> Maybe creating a vlc_object_t for the surface pool on which they would
>> both hold a reference could help ?
>
> I don't know what you think vlc_object_t does. vlc_object_t is a small set of
> parameters for log messages (vlc_messages.h), and a dictionary of values with
> notification callbacks (vlc_variables.h). That is pretty much all there is to
> it. You don't need vlc_object_t for reference counting (e.g. input_item_t).
>
> Reference counting a picture pool makes no sense at this point though. The
> pool is destroyed when its video output or video conversion filter is
> destroyed. Even if the pool were not destroyed, there would be no ways to
> allocate pictures from it. So it might as well be destroyed.
>
> The pictures within the pool are reference counted instead. Pictures will in
> some corner cases outlive their pool.

The surfaces in the picture_t are resources that belong to the D3D9
device. The reference counting of the pool (to find a free one)
doesn't do much about freeing the resources. But I have to dig deeper
there. I may end up just using a simple picture_pool_t that will
allocate surfaces one by one when it needs more. Rather than a large
fixed size I need now. If resource handling is done properly in the
pool then that problem will be solved.

Otherwise the vlc_object_t also has a destructor callback. That could
be useful when neither the decoder nor the vout need the surface pool
and its allocated resources anymore.