[vlc-devel] Display update/restart

[Steve Lhomme]

On 2020-11-16 16:21, Rémi Denis-Courmont wrote:
> Le lundi 16 novembre 2020, 13:24:33 EET Steve Lhomme a écrit :
>> The current way of handling source changes is to redo the filter
>> chain(s) and add an extra filter if the last filter doesn't output what
>> it was supposed to. So the source change is effectively canceled by this
>> last filter.
> 
> That's a very confusing statement. If the decoder output format changes, then
> of course you have to redo the display and the filter chain. (Also what I wrote
> in the previous mail.)

There is no "of course" to redo the display. Many people already 
expressed the possibility of *not* redoing the whole display in that case.

> If the filter chain output changes, then that's the asynchronous filtering
> problem: what if a filter changes its output while running or if the user adds
> a filter than changes the output.
> 
> Currently, the former is outright not permitted, and the later is assumed not
> to occur, and clumsily handled. But they're really two different ways to
> trigger the same issue.

Live format changes from filters is still not a subject. It's not going 
to happen for 4.0. And noone ever expressed that need AFAIK. So let's 
not get lost in moot subjects, there's already enough to discuss.

> The "correct" solution is to instantiate the pipeline from upstream down, i.e.
> spawn the display after the filter, and respawn everything downstream when a
> filter changes. That's more or less what we already do at packetizer and
> decoder levels. For audio, this is not practical because of pass-through. For
> video, I don't know any obvious problem, but there may be.

That seems doable at first glance. But how do you effectively handle 
format changes downstream ? You would need to send the new format in 
case the next filter needs to drain pending pictures. And what happens 
if the next filter rejects the format change ? This is exactly the same 
problems we have to solve for the display.

In other words:
- the video output gets a picture
- the video output sends the picture to the 1st filter
- the 1st filter changes the output format
- the 2nd filter rejects the new output format

 From there we need to drain the 2nd filter before we can actually proceed.
Another option would be to *always* recreate the 2nd filter if the 1st 
filter changes the output, which is what you propose. That still means 
draining. Except the 2nd filter might have been perfectly fine with the 
format change and would have produced better outputs that draining+dry 
start. So not only restarting is slow but it will produce worse output 
than if we handled format changes pushing.

IMO that requires big changes for a feature (filter format change) we 
never discussed and agreed on.

> The simpler hack is to just destroy the display and make a new one, as in the
> previous mail, when the filter chain changes. Now that the picture pool, the
> hardware context and the window are all independent of the display, that's not
> a problem.
> 
> (...)
>> This is becoming critical because we have to handle interlaced HEVC
>> which decoders output field by field, rather than 2 fields at a time for
>> all other interlaced codecs. So when deinterlacing kicks in (patches
>> coming soon), the dimension coming out of filter is double the height.
>> To display it properly we need to update the display or create a new
>> one. We should not use a filter to reduce by half the height and display
>> it in the old display. And given the picture surface is actually double
>> the size, it's quite a big change for the display module.
> 
> I have to disagree here. First, it's not clear at all if we want to support
> interlaced HEVC, because on the face of it, that's really an incredibly stupid
> idea.

It's already in use by some broadcasters so we'll have to live with it.

> And then, the deinterlace resolution problem is not new. We already have some
> deinterlacing algorithms changing the resolution by a factor of 2 (not to
> mention the cropadd filter whose essence is to change the resolution).

Back to square one with the first paragraph that seems to confuse you: 
The current code will cancel such a format change.