[vlc-devel] [PATCH 00/17] Restart the display module on format changes

[Alexandre Janniaux]

Hi,

There seems to be a misunderstanding here.

Romain's point is that OpenGL doesn't enfore a specific
format for the input currently. It uses the interop
infrastructure to have OpenGL textures available, but there
are no conversion at play here: it's either upload (even for
i420) or live binding.

The main performance cost of OpenGL is not really processing
like chroma conversion, but it's directly linked to the number
of rasterization pass and the output format of those pass, of
course excluding demanding additional filters.

There's no issue with binding input i420 too. The «issue» with
i420 in OpenGL is for the output surface. With a native surface,
if it only supports RBGA, then you're stuck with outputting RGBA
at the end of the chain. Additionally, without taking into
account the native visual, outputting i444 is easy, and mostly
available for all platforms we support, but i420 will need at
least two pass to be rasterized, though they don't depend on each
other, and it's much easier (but expensive) to write with three
pass anyway. So you could technically output i420.

The only issue with input i420 and OpenGL is Android native
buffers where you mostly cannot bind each plane independantly.
It's possible with all other platforms we support.

The point of using an OpenGL display at all is to make the
computation on a GPU. If there are no computation to be done,
passthrough display are much more efficient.

That's part of the document I'll publish later, but typically
the goal of OpenGL filters is to move the OpenGL processing
back to the filter_t pipeline, meaning that passthrough display
and transcode pipeline could work with the OpenGL computation
being done outside of their code, avoiding bloating the display
with OpenGL at all.

I feel that the points discussed in this thread are just not
the same, and not on the same use case, so more context and a
more global picture would probably help here.

Regards,
--
Alexandre Janniaux,
Videolabs

On Sun, Nov 22, 2020 at 12:27:16PM +0200, Rémi Denis-Courmont wrote:
> Le sunnuntaina 22. marraskuuta 2020, 12.01.23 EET Romain Vimont a écrit :
> > > No. It's smart. It avoids bloating the display with conversions that would
> > > needed in filters anyway and duplicating code.
> >
> > Hmm, no, this does not add an additional conversion in the display (just
> > a "reconfigure"). In practice, creating the vout after the filters will
> > solve the case where filters output != decoder output.
>
> Yes it does. Add a OpenGL adjust filter operating on OpenGL RGBA textures. How
> is it going to work with a CPU I420 input? It will need conversion. Keeping
> that conversion in the OpenGL display duplicates code and bloats the display
> plugin.
>
> But hey, that was already pointed out at the video workshop. Not that it would
> be the first time that people decide to ignore it.
>
> > > It also avoids breaking the fine
> > > and simple format negotiation and chain build that we have.
> >
> > It's not fine because it causes issues (when a filter produces pictures
> > with chroma or size different from its input picture).
>
> Filter that gratuitiously changes format is broken either way, as that would
> cause unnecessary conversion *between* filters. Building a proper chain is
> never going to work if filters forcing GPU<->CPU transfers, or RGB<->YUV
> conversions.
>
> And then, changing the size is not permissible at all in any case, as it would
> break the windowing code to handle window size and positioning. Whatever
> change of resolution is necessary has to be known up-front, e.g. to handle
> reorientation, *or* an invert SAR adjustment is necessary to compensate.
>
> So those are both bogus scenarii.
>
> > It would be more
> > straightforward to just create the vout display with the format the last
> > filter (if any) produces.
>
> Of course not. That's exactly the wrong thing to do, as it requires each and
> every display to ship their own conversion/import of however many formats,
> *and* it does not play nice with negotiation and filter chain building.
>
> > > > The display sometimes do not have a "most convenient format". For
> > > > example, the OpenGL display doesn't care if it receives I420, NV12 or
> > > > RGBA for example. Adding a converter to match the decoder format (which
> > > > is meaningless) is a pure waste.
> > >
> > > Lol. Native I420 in OpenGL? And you call me stupid?
> >
> > I never said "native in OpenGL", I talked about the vout display.
> > The vout display does not care about the input format, because it always
> > uses an interop to import the picture.
> >
> > Btw, currently, if the decoder produces I420 pictures and the vout
> > display receives RGBA pictures, then a converter from RGBA to I420 must
> > be added so that OpenGL receives I420 (while its "native" format is
> > RGBA).
>
> Self-contradiction much? In your previous mail, I420 was one of the most
> convenient format for OpenGL to receive alongside RGBA, but now it is not
> anymore.
>
> In all my stupidity, I'd think that the OpenGL dispaly wants RGBA GL textures,
> much like the VDPAU wants VDPAU output surfaces, regardless of what comes in.
>
> --
> Реми Дёни-Курмон
> http://www.remlab.net/
>
>
>
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel