[x264-devel] YUY2 to YUV420P with OpenGL ES 3.1

[Henrik Gramner]

On Sat, Jan 25, 2020 at 10:19 PM Chris Hiszpanski <chris at hiszpanski.name> wrote:
> Hello,
>
> Would an OpenGL ES 3.1 compute shader based YUY2 to YUV420P converter be a useful addition to libx264?
>
> I have an NXP i.MX8M based board with a MIPI interface CMOS sensor. There is a v4l2 driver, YUY2 (YUYV) is the only supported pixel format. Yet, a few common decoders/players only support YUV420P. To support these players, a YUY2 to YUV420P (packed to planar) conversion is needed prior to encoding.
>
> First, I tried using a straightforward for-loop (no acceleration beyond compiler optimizations) and achieved 4-5 fps (ffmpeg also produced 4-5 fps).
>
> Next, I tried using ARM intrinsics, which achieved 23-24 fps (but consuming a CPU core).
>
> The i.MX8M has a Vivante GPU, with OpenGL ES 3.1 support, which permits the use of compute shaders. I wrote a shader for YUY2 to YUV420P conversion, encoding the resulting YUV420P frames with libx264 and achieved 59-60 fps, with minimal CPU load.
>
> Would it be useful to include this in libx264, or is it out of scope? How have others handled the YUY2 to YUV420P issue?
>
> Thanks,
> Chris_______________________________________________

Colorspace conversion is a bit out of scope for libx264, since that's
a very deep rabbit hole to go into. Note that YUY2 (YUYV) input is
natively supported for 4:2:2 encoding already, although afaik it only
has SIMD on x86.