[vlc-devel] [PATCH 00/13] OpenGL filters: add support for plane filtering

[Romain Vimont]

Currently, the OpenGL filter API exposes the input VLC picture via a GLSL
function "vec4 vlc_texture(vec2 coords)", which takes care of all required
transformations on-the-fly (swizzle, orientation, offsets, chroma
conversion...) to expose a RGB picture to the fragment shader of the filter.
The filter produces a single RGB output.

 +-------+
 |       | |
 |   Y   | |   vlc_texture()
 |       | |           . . . . .  OpenGL   +-------+
 +-------+ |           .       .  filter   |       |
 +---+     |---------> .  RGB  . ========> |  RGB  |
 | U |     | swizzle   .       .           |       |
 +---+     | orient    . . . . .           +-------+
 +---+     | offsets
 | V |     | chroma
 +---+

This patchset introduces a new feature to filter individual planes instead. If
a filter sets the filter_plane flag, then it is called once per input plane,
and produces as many output textures as there are input textures (and preserves
the chroma). The textures are automatically handled by the OpenGL filter
"engine".

The sampler of the first OpenGL non-plane filter will take care of the chroma
conversion to RGB. If there are no more filters, then a "draw" filter is
inserted automatically to produce the final RGB picture. This requires to add
support for chroma conversion _between_ OpenGL filters (so the chroma
conversion must not depend on the interop anymore).

               vlc_texture()
 +-------+   orient    . . . . .  OpenGL   +-------+    Another OpenGL filter
 |       | | offsets   .       .  filter   |       | |
 |   Y   | |---------> .   Y   . ========> |   Y   | |
 |       | |           .       .           |       | |             +-------+
 +-------+             . . . . .           +-------+ |             |       |
 +---+                 . . .               +---+     |------> ...  |  RGB  |
 | U |     |---------> . U .     ========> | U |     |             |       |
 +---+                 . . .               +---+     |             +-------+
 +---+                 . . .               +---+     |
 | V |     |---------> . V .     ========> | V |     |
 +---+                 . . .               +---+

This feature is useful for deinterlace filters, which operate on individual
planes. The chroma conversion to RGB must be applied _after_ deinterlacing.

Note: The planes are always filtered independently (the filter is applied on
each plane separately). For example, it is not possible to write a plane filter
combining pixels from different planes. This limitation allows to provide a
simple API: for the filter implementation, it's basically just a flag to set.

To test this feature, a new option has been added to the mock filter to apply a
different spatial offset to each plane, so that the chroma planes (for example
if the input is I420) are visually shifted. To run it:

    ./vlc file.mkv --video-filter='opengl{filter=mock{plane}}'

Regards

Branch: https://code.videolan.org/rom1v/vlc/-/commits/filter_plane

Romain Vimont (13):
  opengl: explicit texture scaling computation
  opengl: init swizzle without interop
  opengl: init fragment shader without interop
  opengl: use main shader for direct samplers
  opengl: factorize sampler creation
  opengl: replace multi-variables by arrays in GLSL
  opengl: extract sampler and texture lookup names
  opengl: extract shader extensions
  opengl: expose texture sizes in sampler
  opengl: add support for plane samplers
  opengl: make direct sampler support multi-texture
  opengl: add support for planes filtering
  opengl: add plane filtering to mock filter

 modules/video_output/opengl/filter.c       |   8 +-
 modules/video_output/opengl/filter.h       |  18 +
 modules/video_output/opengl/filter_mock.c  | 129 +++++-
 modules/video_output/opengl/filter_priv.h  |  12 +-
 modules/video_output/opengl/filters.c      | 234 +++++++---
 modules/video_output/opengl/interop.c      |  53 ++-
 modules/video_output/opengl/sampler.c      | 475 +++++++++++++--------
 modules/video_output/opengl/sampler.h      |   7 +
 modules/video_output/opengl/sampler_priv.h |  33 +-
 9 files changed, 698 insertions(+), 271 deletions(-)

-- 
2.30.1