[vlc-devel] [PATCH 3/3] opengl: apply OpenGL vflip from interop
Romain Vimont
rom1v at videolabs.io
Mon Jun 15 19:24:26 CEST 2020
Uploading a picture_t to OpenGL flips it vertically.
To display the video in the correct orientation, the renderer reversed
the transformation: each vertex received as attribute texture
coordinates (x, 1-y) instead of (x, y), in order to flip the image
vertically.
This caused several problems.
Firstly, the renderer must be independent of the input picture storage.
Otherwise, when it receives an input in the normal orientation, it will
wrongly apply a vertical flip.
Secondly, since the vflip was applied on the input coordinates, it
occurred before the orientation transform:
OrientationMatrix * VFlipMatrix * input_coords (semantically)
whereas the correct transformation is:
VFlipMatrix * OrientationMatrix * input_coords
(for reasons explained in e329c4bbce5ad080e20727e6a0db5f08568aee43).
Since matrix multiplication is not commutative, it resulted in a
wrong orientation in some cases (for example if OrientationMatrix
contains a rotation by 90 degrees).
(In fact, in pratice, this case still worked, because the initialization
of OrientationMatrix was also wrong, see the two previous commits).
To fix all these problems, initialize the texture coordinates in the
normal orientation in the renderer, and apply the vertical flip on the
interop format orientation.
This also allows to simplify the Android interop, which can just provide
its own transform matrix without compensating for the vertical flip
that was applied in the renderer.
---
modules/video_output/opengl/interop_android.c | 31 -----------
modules/video_output/opengl/interop_cvpx.c | 6 +++
modules/video_output/opengl/interop_dxva2.c | 3 ++
modules/video_output/opengl/interop_sw.c | 3 ++
modules/video_output/opengl/interop_vaapi.c | 3 ++
modules/video_output/opengl/interop_vdpau.c | 3 ++
modules/video_output/opengl/renderer.c | 51 ++++++++++---------
7 files changed, 44 insertions(+), 56 deletions(-)
diff --git a/modules/video_output/opengl/interop_android.c b/modules/video_output/opengl/interop_android.c
index 29fef17fce97..aee9766884e1 100644
--- a/modules/video_output/opengl/interop_android.c
+++ b/modules/video_output/opengl/interop_android.c
@@ -137,37 +137,6 @@ Open(vlc_object_t *obj)
};
interop->ops = &ops;
- /* The transform Matrix (uSTMatrix) given by the SurfaceTexture is not
- * using the same origin than us. Ask the caller to rotate textures
- * coordinates, via the vertex shader, by forcing an orientation. */
- switch (interop->fmt.orientation)
- {
- case ORIENT_TOP_LEFT:
- interop->fmt.orientation = ORIENT_BOTTOM_LEFT;
- break;
- case ORIENT_TOP_RIGHT:
- interop->fmt.orientation = ORIENT_BOTTOM_RIGHT;
- break;
- case ORIENT_BOTTOM_LEFT:
- interop->fmt.orientation = ORIENT_TOP_LEFT;
- break;
- case ORIENT_BOTTOM_RIGHT:
- interop->fmt.orientation = ORIENT_TOP_RIGHT;
- break;
- case ORIENT_LEFT_TOP:
- interop->fmt.orientation = ORIENT_RIGHT_TOP;
- break;
- case ORIENT_LEFT_BOTTOM:
- interop->fmt.orientation = ORIENT_RIGHT_BOTTOM;
- break;
- case ORIENT_RIGHT_TOP:
- interop->fmt.orientation = ORIENT_LEFT_TOP;
- break;
- case ORIENT_RIGHT_BOTTOM:
- interop->fmt.orientation = ORIENT_LEFT_BOTTOM;
- break;
- }
-
int ret = opengl_interop_init(interop, GL_TEXTURE_EXTERNAL_OES,
VLC_CODEC_RGB32,
COLOR_SPACE_UNDEF);
diff --git a/modules/video_output/opengl/interop_cvpx.c b/modules/video_output/opengl/interop_cvpx.c
index 7c57455d788b..be3d9cecce9b 100644
--- a/modules/video_output/opengl/interop_cvpx.c
+++ b/modules/video_output/opengl/interop_cvpx.c
@@ -173,6 +173,9 @@ Open(vlc_object_t *obj)
&& interop->fmt.i_chroma != VLC_CODEC_CVPX_P010)
return VLC_EGENERIC;
+ /* The pictures are uploaded upside-down */
+ video_format_TransformBy(&interop->fmt, TRANSFORM_VFLIP);
+
struct priv *priv = calloc(1, sizeof(struct priv));
if (unlikely(priv == NULL))
return VLC_ENOMEM;
@@ -211,6 +214,9 @@ Open(vlc_object_t *obj)
}
#endif
+ /* The pictures are uploaded upside-down */
+ video_format_TransformBy(&interop->fmt, TRANSFORM_VFLIP);
+
int ret;
switch (interop->fmt.i_chroma)
{
diff --git a/modules/video_output/opengl/interop_dxva2.c b/modules/video_output/opengl/interop_dxva2.c
index cf2a7a756380..2b0f2811cb23 100644
--- a/modules/video_output/opengl/interop_dxva2.c
+++ b/modules/video_output/opengl/interop_dxva2.c
@@ -509,6 +509,9 @@ GLConvOpen(vlc_object_t *obj)
};
interop->ops = &ops;
+ /* The pictures are uploaded upside-down */
+ video_format_TransformBy(&interop->fmt, TRANSFORM_VFLIP);
+
int ret = opengl_interop_init(interop, GL_TEXTURE_2D, VLC_CODEC_RGB32,
COLOR_SPACE_UNDEF);
if (ret != VLC_SUCCESS)
diff --git a/modules/video_output/opengl/interop_sw.c b/modules/video_output/opengl/interop_sw.c
index a553f4f2b4f5..5571d6bcef43 100644
--- a/modules/video_output/opengl/interop_sw.c
+++ b/modules/video_output/opengl/interop_sw.c
@@ -321,6 +321,9 @@ opengl_interop_generic_init(struct vlc_gl_interop *interop, bool allow_dr)
space = COLOR_SPACE_UNDEF;
}
+ /* The pictures are uploaded upside-down */
+ video_format_TransformBy(&interop->fmt, TRANSFORM_VFLIP);
+
int ret = VLC_EGENERIC;
while (*list)
{
diff --git a/modules/video_output/opengl/interop_vaapi.c b/modules/video_output/opengl/interop_vaapi.c
index bee10624e3f7..b1ed1bd95a5e 100644
--- a/modules/video_output/opengl/interop_vaapi.c
+++ b/modules/video_output/opengl/interop_vaapi.c
@@ -482,6 +482,9 @@ Open(vlc_object_t *obj)
if (tc_va_check_derive_image(interop))
goto error;
+ /* The pictures are uploaded upside-down */
+ video_format_TransformBy(&interop->fmt, TRANSFORM_VFLIP);
+
int ret = opengl_interop_init(interop, GL_TEXTURE_2D, vlc_sw_chroma,
interop->fmt.space);
if (ret != VLC_SUCCESS)
diff --git a/modules/video_output/opengl/interop_vdpau.c b/modules/video_output/opengl/interop_vdpau.c
index 35f622235154..9b0d3adf31a3 100644
--- a/modules/video_output/opengl/interop_vdpau.c
+++ b/modules/video_output/opengl/interop_vdpau.c
@@ -176,6 +176,9 @@ Open(vlc_object_t *obj)
INTEROP_CALL(glVDPAUInitNV, (void *)(uintptr_t)device, vdp_gpa);
+ /* The pictures are uploaded upside-down */
+ video_format_TransformBy(&interop->fmt, TRANSFORM_VFLIP);
+
int ret = opengl_interop_init(interop, GL_TEXTURE_2D, VLC_CODEC_RGB32,
COLOR_SPACE_UNDEF);
if (ret != VLC_SUCCESS)
diff --git a/modules/video_output/opengl/renderer.c b/modules/video_output/opengl/renderer.c
index 6a778ce35023..1419927891ca 100644
--- a/modules/video_output/opengl/renderer.c
+++ b/modules/video_output/opengl/renderer.c
@@ -488,7 +488,8 @@ static int BuildSphere(GLfloat **vertexCoord, GLfloat **textureCoord, unsigned *
unsigned off2 = (lat * (nbLonBands + 1) + lon) * 2;
float u = (float)lon / nbLonBands;
- float v = (float)lat / nbLatBands;
+ /* In OpenGL, the texture coordinates start at bottom left */
+ float v = 1.0f - (float)lat / nbLatBands;
(*textureCoord)[off2] = u;
(*textureCoord)[off2 + 1] = v;
}
@@ -577,35 +578,35 @@ static int BuildCube(float padW, float padH,
float row[] = {0.f, 1.f/2, 1.0};
const GLfloat tex[] = {
- col[1] + padW, row[1] + padH, // front
- col[1] + padW, row[2] - padH,
- col[2] - padW, row[1] + padH,
- col[2] - padW, row[2] - padH,
-
- col[3] - padW, row[1] + padH, // back
- col[3] - padW, row[2] - padH,
- col[2] + padW, row[1] + padH,
- col[2] + padW, row[2] - padH,
-
- col[2] - padW, row[0] + padH, // left
- col[2] - padW, row[1] - padH,
+ col[1] + padW, row[1] - padH, // front
col[1] + padW, row[0] + padH,
- col[1] + padW, row[1] - padH,
+ col[2] - padW, row[1] - padH,
+ col[2] - padW, row[0] + padH,
- col[0] + padW, row[0] + padH, // right
- col[0] + padW, row[1] - padH,
- col[1] - padW, row[0] + padH,
- col[1] - padW, row[1] - padH,
+ col[3] - padW, row[1] - padH, // back
+ col[3] - padW, row[0] + padH,
+ col[2] + padW, row[1] - padH,
+ col[2] + padW, row[0] + padH,
+
+ col[2] - padW, row[2] - padH, // left
+ col[2] - padW, row[1] + padH,
+ col[1] + padW, row[2] - padH,
+ col[1] + padW, row[1] + padH,
- col[0] + padW, row[2] - padH, // bottom
+ col[0] + padW, row[2] - padH, // right
col[0] + padW, row[1] + padH,
col[1] - padW, row[2] - padH,
col[1] - padW, row[1] + padH,
- col[2] + padW, row[0] + padH, // top
- col[2] + padW, row[1] - padH,
- col[3] - padW, row[0] + padH,
- col[3] - padW, row[1] - padH,
+ col[0] + padW, row[0] + padH, // bottom
+ col[0] + padW, row[1] - padH,
+ col[1] - padW, row[0] + padH,
+ col[1] - padW, row[1] - padH,
+
+ col[2] + padW, row[2] - padH, // top
+ col[2] + padW, row[1] + padH,
+ col[3] - padW, row[2] - padH,
+ col[3] - padW, row[1] + padH,
};
memcpy(*textureCoord, tex,
@@ -658,10 +659,10 @@ static int BuildRectangle(GLfloat **vertexCoord, GLfloat **textureCoord, unsigne
memcpy(*vertexCoord, coord, *nbVertices * 3 * sizeof(GLfloat));
static const GLfloat tex[] = {
- 0.0, 0.0,
0.0, 1.0,
- 1.0, 0.0,
+ 0.0, 0.0,
1.0, 1.0,
+ 1.0, 0.0,
};
memcpy(*textureCoord, tex, *nbVertices * 2 * sizeof(GLfloat));
--
2.27.0
More information about the vlc-devel
mailing list