[vlc-devel] [PATCH] swscale: Fix pixel format for VLC_CODEC_RGBA

[Rémi Denis-Courmont]

Le jeudi 20 décembre 2012 19:45:14, Florian Albrechtskirchinger a écrit :
> > RGBA is: R at byte 0, G at byte 1, B at byte 2, A at byte 3.
> 
> To reiterate:
> On my system the screen capture chroma is either RGB32 or RGBA,

X.Org uses BGRA on little endian systems, rather than RGBA. At least both the 
ATI and NVIDIA drivers do:

  visual:
    visual id:    0x76
    class:    TrueColor
    depth:    32 planes
    available colormap entries:    256 per subfield
    red, green, blue masks:    0xff0000, 0xff00, 0xff
    significant bits in color specification:    8 bits

> depending on the window. In both cases Blue is at byte 0 and Red at byte 2.
> By your definition the chromas should be BGR32 and BGRA respectively.

Yes and no. For historical reasons, VLC_CODEC_RGB32 (RV32) represents all 24-
in-32 bits RGB pixel formarts. To specify the colour position, i_(r|g|b)mask 
are used. Retrospectively, I believe that was a bad idea (not mine!); I think 
the libav approach is better/clearer.

VLC_CODEC_RGBA represents *only* RGBA. For X11 capture, a new codec needs to 
be defined. Alternatively, the alpha mask could be discarded completely.

-- 
Rémi Denis-Courmont
http://www.remlab.net/