[vlc-devel] [PATCH 00/41] Abstract picture storage from renderer

[Romain Vimont]

The role of the renderer is to draw the input pictures using shaders. The
rendering depends on several properties, like the multiview mode (stereo), the
projection mode (rectangular, equirectangular, cubemap) and the viewpoint.

But the renderer had to deal with the details related to the storage of the
input pictures:
 - the interop
 - the number of planes and the textures coordinates for each plane
 - the actual orientation (video_orientation_t and transform matrix from
   Android)
 - the paddings (due to crop or power-of-2 textures)
 - the update of textures from a picture_t

The goal of this patchset is to handle all these details from a separate
component, called a "sampler".

The sampler exposes a GLSL function for the fragment shader:

    vec4 vlc_texture(vec2 coords)

which returns the RGBA values for the given coordinates. It behaves like the
standard GLSL function:

    vec4 texture2D(sampler2D sampler, vec2 coords)

except that the input picture is implicit (so there is no explicit sampler2D).
It takes care of the swizzle, chroma conversion, etc.

The renderer "injects" the code generated from the sampler into its fragment
shader.

The old code is transformed step by step without rewriting the logic (except
for paddings, which required more work to split the responsibilities, see
patches 6 and 9).

In particular, the code for handling chroma conversion has been kept, and
should work as before. It is now just "injectable" into any fragment shader.

In addition to improve the separation of concerns, it paves the way to
implement OpenGL filters, which could read the input pictures from VLC without
dealing with how they are stored.

This patchset is based on this previous one (not merged yet):

    https://mailman.videolan.org/pipermail/vlc-devel/2020-February/130958.html

The original branch is available at:

    https://code.videolan.org/rom1v/vlc/commits/glsampler

Romain Vimont (41):
  opengl: factorize program creation
  opengl: move vtable to vlc_gl_api
  opengl: pass vlc_gl_api instead of virtual table
  opengl: move fields from interop to gl_api
  opengl: expose npot support in gl_api
  opengl: convert texture coords in fragment shader
  opengl: setup coords once for all
  opengl: apply orientation in fragment shader
  opengl: apply stereo transform using a matrix
  opengl: merge successive loops
  opengl: move down vlc_gl_renderer_Prepare()
  opengl: move paddings computation to _Prepare()
  opengl: remove unused source format parameter
  opengl: extract vlc_texture()
  opengl: move fragment shader creation to renderer
  opengl: extract renderer fields to sampler
  opengl: move interop ownership to vgl
  opengl: use the interop format in renderer
  opengl: reference interop from sampler
  opengl: extract sampler creation and destruction
  opengl: reference common objects from sampler
  opengl: define identity matrices in gl_util.h
  opengl: move callbacks to sampler
  opengl: fetch locations from sampler
  opengl: load uniforms from sampler
  opengl: remove alpha parameter from prepare_shader
  opengl: simplify pf_prepare_shader()
  opengl: expose functions to call sampler callbacks
  opengl: remove interop reference from renderer
  opengl: only use sampler from fragment_shaders.c
  opengl: store fragment shader in sampler
  opengl: generate the extensions from the sampler
  opengl: move sampler initialization to constructor
  opengl: move orientation matrix init to sampler
  opengl: update pictures from sampler
  opengl: expose video format in sampler
  opengl: move sampler ownership to vgl
  opengl: remove all interop usages from renderer
  opengl: do not pass the picture_t to the renderer
  opengl: merge DrawWithShaders() into _Draw()
  opengl: merge fragment_shaders.c into sampler.c

 modules/video_output/Makefile.am              |   7 +-
 modules/video_output/opengl/display.c         |   2 +-
 .../video_output/opengl/fragment_shaders.c    | 667 -------------
 modules/video_output/opengl/gl_api.c          | 154 +++
 modules/video_output/opengl/gl_api.h          |  50 +
 modules/video_output/opengl/gl_util.c         | 132 +++
 modules/video_output/opengl/gl_util.h         |  35 +
 modules/video_output/opengl/internal.h        |   3 -
 modules/video_output/opengl/interop.c         |  16 +-
 modules/video_output/opengl/interop.h         |  11 +-
 modules/video_output/opengl/interop_sw.c      |   7 +-
 modules/video_output/opengl/interop_vaapi.c   |   3 +-
 modules/video_output/opengl/interop_vdpau.c   |   3 +-
 modules/video_output/opengl/renderer.c        | 735 +++++---------
 modules/video_output/opengl/renderer.h        |  88 +-
 modules/video_output/opengl/sampler.c         | 939 ++++++++++++++++++
 modules/video_output/opengl/sampler.h         | 181 ++++
 modules/video_output/opengl/sub_renderer.c    | 185 +---
 modules/video_output/opengl/sub_renderer.h    |   8 +-
 modules/video_output/opengl/vout_helper.c     | 187 ++--
 modules/video_output/opengl/vout_helper.h     |   3 +-
 modules/video_output/win32/glwin32.c          |   2 +-
 22 files changed, 1878 insertions(+), 1540 deletions(-)
 delete mode 100644 modules/video_output/opengl/fragment_shaders.c
 create mode 100644 modules/video_output/opengl/gl_api.c
 create mode 100644 modules/video_output/opengl/gl_api.h
 create mode 100644 modules/video_output/opengl/gl_util.c
 create mode 100644 modules/video_output/opengl/sampler.c
 create mode 100644 modules/video_output/opengl/sampler.h

-- 
2.25.0