[vlc-devel] [PATCH] core: add a callback to init/release data for picture pool of opaque formats

[Steve Lhomme]

On Mon, Apr 20, 2015 at 5:50 PM, Rémi Denis-Courmont <remi at remlab.net> wrote:
> Le lundi 20 avril 2015, 17:22:07 Steve Lhomme a écrit :
>> > So I don't see how non-serial data in video_format_t (or alongside it) can
>> > ever work. FWIW, that issue was already brought up to solve a completely
>> > different problem - multiple video output windows.
>> >
>> > To take a simple and concrete example, how do you compare the video
>> > formats
>> > when there is an opaque pointer inside? Pointer comparison only works if
>> > you know that the two pointed objects were concurrent in memory, such
>> > that they cannot be two distinct objects with the same address.
>>
>> There's one case where the code above may not work. If the old
>> decoder_pool_setup_sys_t is freed, a new one is reallocated in the
>> exact same place. Then it wouldn't work.
>
> That is the point: Do not put non-serial(izable) data, such as opaque
> pointers, into or alongside format structures. It is not safe.
>
>> I'm open to solutions to solve this.
>
> Well, I can only repeat what I already wrote:
>
> The *proper* solution is to pull any resources you need. Currently it can be
> done with the picture_sys_t, if the chroma is custom *and* if there are no
> video filters in the chain. Ideally though, there could add a function for the
> decoder to retrieve the hardware context from the video output after updating
> the output format but without pulling any specific picture.

In the case of DXVA that would mean delaying the surface allocation
until the vout can provide it. Why not. But what happens if there's no
compatible vout ?

That sounds like doing the whole decoding/displaying chain backwards
and that would open a whole big can of worms.

Anyway it cannot work with DXVA because, if I understand the chaining
properly, the vout is created when the first frame comes out of the
decoder. So it needs decoding surfaces before there's a vout.

> Other than that, you have the kludges used by the preexisting opaque formats,
> involving the decoder writing into the picture_t context, the picture_sys_t
> and/or global variables.

I'll look further into this to see what tricks are used. For example
the Android vout seem to know about the decoder_t and the decoder
knows when it's being used for direct rendering.