[vlc-devel] [PATCH] opengl: use common fragment shader for Android

[Romain Vimont]

On Thu, Dec 19, 2019 at 09:22:22PM +0800, zhilizhao wrote:
> 
> 
> > On Dec 19, 2019, at 8:58 PM, Romain Vimont <rom1v at videolabs.io> wrote:
> > 
> > On Thu, Dec 19, 2019 at 08:15:23PM +0800, zhilizhao wrote:
> >> Hi Romain,
> >> 
> >>> On Dec 19, 2019, at 6:32 PM, Romain Vimont <rom1v at videolabs.io> wrote:
> >>> @@ -315,12 +322,13 @@ static GLuint BuildVertexShader(const opengl_tex_converter_t *tc,
> >>>        "attribute vec4 MultiTexCoord0;\n"
> >>>        "%s%s"
> >>>        "attribute vec3 VertexPosition;\n"
> >>> +        "uniform mat4 TransformMatrix;\n"
> >>>        "uniform mat4 OrientationMatrix;\n"
> >>>        "uniform mat4 ProjectionMatrix;\n"
> >>>        "uniform mat4 ZoomMatrix;\n"
> >>>        "uniform mat4 ViewMatrix;\n"
> >>>        "void main() {\n"
> >>> -        " TexCoord0 = vec4(OrientationMatrix * MultiTexCoord0).st;\n"
> >>> +        " TexCoord0 = vec4(TransformMatrix * OrientationMatrix * MultiTexCoord0).st;\n”
> >> 
> >> TransformMatrix requires the texture coordinate has the form of (s, t, 0, 1). Does the condition satisfied in this case?
> > 
> > Good question :)
> > 
> > Yes, I think it is always satisfied.
> > 
> > Given that MultiTexCoord0 represent (2D) texture coordinates, it is
> > always in the form [x, y, 0, 1].
> > 
> > OrientationMatrix is initialized in getOrientationTransformMatrix(). In
> > this function, the matrix is first initialized to identity _(oh, there
> > was already an "identity" constant, I will reuse it)_, then some
> > components are overwritten. Note that in memory, it is stored in
> > column-major order, so indices initialization must be intrepreted as:
> > 
> >    matrix[col * 4 + row] = value;
> > 
> > We see that row 3 is never overwritten, so the last row of the
> > orientation matrix is necessarily [0, 0, 0, 1], and the last component
> > of the output is 1.
> > 
> > Since MultiTexCoord0[2] == 0 (2D input coordinates, z == 0), row 2 is
> > irrelevant, and the third component of the output is 0.
> > 
> > Therefore, OrientationMatrix * MultiTexCoord0 is always in the form
> > [x', y', 0, 1] (which is the input of the TransformMatrix).
> > 
> > More intuitively, OrientationMatrix only represent 2D linear
> > transformations (rotation, flip, transpose…) of 2D vectors (texture
> > coordinates).
> 
> Thank you for such comprehensive explanation! I forgot matrix in
> OpenGL is column-major by default, every time.

(It also depends on whether you pass GL_FALSE or GL_TRUE for the
"transpose" parameter in glUniformMatrix4fv().)

> TransformMatrix on Android is basically do the same thing as
> orientationMatrix, plus crop.

Yes, both are "orientation matrices". But we need both:
 - OrientationMatrix is computed by us from fmt->orientation;
 - TransformMatrix is provided as-is by Android (it's an additional
   orientation transformation to apply).