[vlc-devel] [PATCH 1/6] decoder: add code to send 3d metadata to the output modules to be handled

[Steve Lhomme]

On 14/08/2018 17:05, Rémi Denis-Courmont wrote:
> Also testing the other way around I found out that changing the vout
>> dynamically like that fails the hardware decoder. In D3D the context is
>> created by the vout and shared with the VA, if the vout disappears and
>> the VA remains unchanged the new vout will have a different context and
>> it won't work (black screen).
> If you change the video output behind the back of the decoder, then it won't
> work.
>
> But as far as I understand this patch, this is about the case of the decoder
> proactively changing its output format. In that case, the decoder is
> responsible for using picture buffers from the picture pool of the new video
> output. Nothing else can fix or work around it for the decoder.
>
Actually it's not a change from the decoder but "side data" that changed 
in the stream. See DecodeSidedata() in avcodec/video.c.

More generally I think we should differentiate the actual picture format 
(as in pixels needed to decode, the get_format from lavc) from 
'metadata' changes including the offset/visible area, sar, orientation, 
HDR, stereo source, etc. In the first case if the chroma or pixels 
change we need a new vout. In all the other cases it depends on the vout 
capability and since it's not requested by the decoder and will require 
a decoder restart we should do our best not to recreate the vout.

This is also related to this thread 
http://ffmpeg.org/pipermail/ffmpeg-devel/2018-April/228842.html 
get_format is currently abused by assuming it will be called when all 
kinds of metadata will change in the source but that's not what it's for 
and we shouldn't use it that way.

Also all these updates depend on the picture being decoded (you don't 
want the change applied too soon) so it's better if they are just passed 
on with the picture (they are) and the vout dynamically handle the 
changes. It may be handled in the core so if the changes is too slow we 
can drop frames accordingly (switch 2d<>3d is not instantaneous).

To do that we need to know beforehand if the vout is capable of doing 
that or not, see "[RFC] video_output: keep track of the vout modules 
that can switch stereo modes dynamically". It should be opt-in so we can 
gradually add the functionalities without breaking anything.