[vlc-devel] [PATCH 1/5] vout: update the format after filters

[Romain Vimont]

On Thu, Oct 22, 2020 at 08:31:11AM +0300, Rémi Denis-Courmont wrote:
> Le keskiviikkona 21. lokakuuta 2020, 19.30.31 EEST Romain Vimont a écrit :
> > On Wed, Oct 21, 2020 at 07:12:48PM +0300, Rémi Denis-Courmont wrote:
> > > Le keskiviikkona 21. lokakuuta 2020, 18.50.51 EEST Romain Vimont a écrit :
> > > > On Wed, Oct 21, 2020 at 05:26:01PM +0300, Rémi Denis-Courmont wrote:
> > > > > Le keskiviikkona 21. lokakuuta 2020, 15.51.03 EEST Romain Vimont a 
> écrit :
> > > > > > Meanwhile, this patchset is far less ambitious (and quite
> > > > > > independent),
> > > > > > it's just a local optimization/fix: instead of inserting a converter
> > > > > > unconditionally on filters/vout format mismatch, the core just
> > > > > > requests
> > > > > > the vout to accept the new format directly. If it can't, it just
> > > > > > inserts
> > > > > > a converter like before, but if it can, this avoids an additional
> > > > > > converter (and a failure for opaque formats).
> > > > > 
> > > > > And yet it still breaks in the ways that were already pointed out when
> > > > > this
> > > > > was proposed and rejected at the winter 2019 workshop.
> > > > 
> > > > I'm sorry, I don't understand how it breaks.
> > > > 
> > > > It just adds a possibility for the vout to accept a format change due to
> > > > filters, in order to avoid the insertion of a converter. If the vout
> > > > refuses, a converter is inserted as before.
> > > 
> > > That's pretty convoluted just for one single display module and naughty
> > > filters. Normally, we take the output of the previous filter and give that
> > > as input to the next, which is both simple and locally optimal.
> > > 
> > > I don't see why we should create a display with the wrong format, and then
> > > change the format.
> > 
> > Because currently, we create a display with the wrong format, and then
> > insert a converter. That's not better.
> 
> There's no such thing as creating a display with the wrong format, unless 
> either a display or a filter messes it up.

A filter producing a picture in a format different from the source is
not "messing up". By the way, if the input is in memory and the filter
is GPU, the output might be an opaque format (and you obviously don't
want to insert a converter before the vout).

> The display is free to request a 
> better format than what the decoder gives, and indeed, many display plugins do 
> so, notably GPU-accelerated one.
> 
> > > For obvious performance reasons, filters should not change the format.
> > 
> > That's not true for GPU filters.
> 
> That's of course true for all types of filters.

No, that's not true for GPU filters.

For example, producing an I420 output could require to execute a filter
3 times, one for each output plane, to 3 different textures.

> Gratuitious conversions may be 
> passable in audio output, but they're a massive wastage of computing resource 
> in video.

That's one of the reason we want to avoid adding an useless converter
and request the vout to accept the new format directly.

One other reason is to keep the result of the filter in GPU to avoid
downloading it to memory if it's not necessary (for obvious performance
reasons).

Regards