[vlc-devel] [PATCH 1/6] decoder: add code to send 3d metadata to the output modules to be handled

[Steve Lhomme]

On 20/08/2018 17:15, Rémi Denis-Courmont wrote:
> The problem is, invalidating pictures breaks direct rendering. We can 
> always
> discuss whether some legacy display plugins could be switched to indirect
> rendering, or removed. However the OpenGL display cannot invalidate pictures.
> And so any such property must be supported by the OpenGL display at the very
> least.
>

For now we only support Side By Side, Top-Bottom and Frame Sequential. 
And OpenGL already supports SBS and TB on the fly:
http://git.videolan.org/?p=vlc.git;a=blob;f=modules/video_output/opengl/vout_helper.c;h=33f738f57928f932312c68064d0cc7a990574bd0;hb=HEAD#l1629

It's called for every picture being displayed. It should be adapted to 
select the right eye when it's merged but that's a good start.

So that leaves Frame Sequential handling. It's not up to the vout 
display to do the decimation otherwise pausing may end up providing one 
eye that is actually not supposed to be shown. In D3D11 a reference is 
kept to the other eye when displaying but it should be moved to the 
core. So moving this logic in the core (keep one eye from sequential 
until we get the next one and chain them using picture->next) might also 
allow doing the decimation.

We would not need any invalid picture in OpenGL to do this. And none of 
that breaks direct rendering.


I tried the invalid pictures solution for other vouts but I couldn't 
find a proper way to do it. Even the non-dynamic case is not trivial. 
For example if a SBS file is opened it's marked from the start so will 
be opened like that. Since we know the vout should handle the 
crop/aspect ratio we give it on startup, we could tweak these parameters 
to display only one eye. But we only know if the module can handle 
Stereoscopy silently (OpenGL, D3D11) or not once the module is opened. 
So it cannot be passed on init but tweaked afterwards. This could result 
in invalid pictures. We could have a vout_display helper that computes 
the crop/aspect ratio for a SBS/TB source when the vout can't handle it 
silently. It would be up to each vout to call it on startup and when the 
source crop/aspect ratio changes.

On the other hand the invalid pictures (reset the pool and converter 
filters) creates other problems. It's up to each vout to decide the 
vd->fmt they want based on the vd->source and config parameters. The 
core cannot modify it, just add converters between the source and fmt. 
And what we don't want is doing picture copies when all we need is a 
crop and a stretch of the source image. But we can't even set the aspect 
ratio in in vd->fmt the i_sar_num/i_sar_den of the fmt are initialized 
to 0 and never used. So we can't double the dimensions for SBS/TB.


So here is what I propose:
- handle Frame Sequential in the core (pairing left and right picture_t 
and decimate the eye we don't want when not in stereoscopy)
- for SBS/TB let the vout tell if it can handle it or not during Open() 
(currently OpenGL)
- when SBS/TB is not handled, the core generates the necessary 
CHANGE_SOURCE_CROP/AR. In most cases there won't be a 
vout_display_SendEventPicturesInvalid() call. It will with vdpau but I 
don't know if it breaks direct rendering in that case.