[vlc-devel] Display update/restart

[Steve Lhomme]

On 2020-11-16 12:24, Steve Lhomme wrote:
> One of the last missing part in push is the possibility to 
> update/restart the display module when the source changes. A corollary 
> of this is when we decide to use a converter filter or not.
> 
> The current way of handling source changes is to redo the filter 
> chain(s) and add an extra filter if the last filter doesn't output what 
> it was supposed to. So the source change is effectively canceled by this 
> last filter.
> 
> It's even worse because instead of updating osys->converter we add an 
> extra filter to the interactive chain. And then there will still be the 
> old osys->convereter. I have patches to fix this and update the 
> osys->converter instead (the extra added filter never needed to be 
> "interactive").
> 
> No matter what, we will still need the osys->converter because not all 
> display modules can handle all chroma/colorimetry/360 formats we send them.
> 
> So when a new format should we
> - create a new display no matter what ?
> - reuse the display if possible ? And if not create a new display ?
> 
>  From previous mail it doesn't seem clear what we should do. It seems 
> Remi is leaning towards #1 while Alexandre, Romain and myself towards #2.
> 
> This is becoming critical because we have to handle interlaced HEVC 
> which decoders output field by field, rather than 2 fields at a time for 
> all other interlaced codecs. So when deinterlacing kicks in (patches 
> coming soon), the dimension coming out of filter is double the height. 
> To display it properly we need to update the display or create a new 
> one. We should not use a filter to reduce by half the height and display 
> it in the old display. And given the picture surface is actually double 
> the size, it's quite a big change for the display module.
> 
> Even if we try to reuse the display with the new format, there will 
> always be cases where the display refuses. So we have to handle the 
> restart/recreate no matter what.
> 
> There is another issue arising from this. We only know about this format 
> change after the picture is deinterlaced. With the current code, that 
> happens when preparing (pre-rendering) a picture. Except this is when 
> deinterlacing the picture.next which is not the one that will be sent to 
> the display right away (picture.current). So this is not the right 
> moment to restart the display. We may need to change how we handle this.

There is also an issue with the frame dropping. If you have 1 field per 
picture and you drop 1 picture the next picture given to the 
deinterlacer will not be correct, it will be the same top (or bottom) 
field. Then subsequent pictures merged will have the fields upside/down.

I think the picture dropping should occur outside of the "pre-rendering" 
to avoid this. If a picture is late it will still be late after that. We 
will just waste some CPU/GPU merging an deinterlacing the picture.

Alternatively we should signal if the single field is top or bottom with 
each field/picture. The deinterlacing will not mix up the field order in 
the merged fields. With a long drop of pictures it may merge a very old 
field with a new picture, so to avoid producing bogus output we should 
also notify the deinterlacer (so filter and filter chain in general) if 
a discontinuity occured when pushing a new picture. With a drain in 
place (see patchset already proposed) that would mean draining the 
filter (ie we're not going to feed it new fields for a while). It may 
decide to drop the single field or double the height of each pixel for 
the last picture.

> I think it is better to reuse the display module if possible. In the end 
> people are only going to use opengl/direct3d11 99% of the time and they 
> are capable of handling such texture size changes, even without 
> recompiling shaders. It's a lot less time sensitive as well. We don't 
> have to worry if we'll have time to restart a display to display the 
> frame that has the new format.
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel