[vlc-devel] Re: Fwd: [Possible endian bug in video_chrome/video_output]
gall at switch.ch
Mon Apr 3 15:47:21 CEST 2006
[ I posted this message yesterday but I didn't see it appear on the
list, even though our MTA delivered it to krishna.via.ecp.fr. Can the
list managers please check what's going on? It has happened before.]
On Sat, 1 Apr 2006 01:27:49 +0200, Derk-Jan Hartman <hartman at videolan.org> said:
> Could you please check the values that are used in
> #define RGB2PIXEL( p_vout, i_r, i_g, i_b ) \
> (((((uint32_t)i_r) >> p_vout->output.i_rrshift) \
> << p_vout->output.i_lrshift) \
> | ((((uint32_t)i_g) >> p_vout->output.i_rgshift) \
> << p_vout->output.i_lgshift) \
> | ((((uint32_t)i_b) >> p_vout->output.i_rbshift) \
> << p_vout->output.i_lbshift))
> on your machine. Sam suspects your problem is in those values.
The mask values are wrong. They should be
I'm not sure where exactly these are set for opengl. If it's in
modules/video_output/opengl.c, then the problem is obvious, because
the values are just set statically, instead of read from the
capabilities of the output device.
Now, when I set the proper mask values by hand, the output is still
not ok. This is because src/video_output/video_output.c:MaskToShift()
produces the wrong result for a mask of 0xff000000. The line
i_high = i_mask + i_low; /* higher bit of the mask */
results in an overflow and the value of rshift becomes 31369 (it
should be 0).
When I fix this as well, the output is finally correct.
BTW, MaskToShift seems to expect that the mask can be smaller than 8
bits, in which case the lower order bits of a color value are
discarded by RGB2PIXEL. I'm wondering whether this makes sense and if
it actually occurs in practice.
Finally, the same problem needs to be fixed for the xvideo and glx
output modules as well.
This is the vlc-devel mailing-list, see http://www.videolan.org/vlc/
To unsubscribe, please read http://developers.videolan.org/lists.html
More information about the vlc-devel