[vlc-devel] RGB capture issues in DirectShow

[Salah-Eddin Shaban]

Hello,

I'm trying to fix two issues in RGB DShow capture. The first one is
related to orientation and the second to chroma.

The orientation issue is that frames are bottom to top, and should be
flipped before rendering. But the statement in line 720 of dshow.cpp

fmt.video.orientation = ORIENT_BOTTOM_LEFT;

is interpreted differently by different vout modules. When using
direct3d11 it seems to have no effect (so frames are still rendered
upside down). With direct3d9 it causes horizontal (not vertical)
flipping, so the end result looks like this

https://drive.google.com/open?id=0B36ioujDBJZsZ0w1cDIyTzJkUjg

With --vout=opengl however it is rendered correctly.

So this is not a DShow bug right?
In which case #11778 should be fixed in the direct3d vout modules.

The chroma issue is swapped red and blue. This happens with all
capture sources that use RGB24 output. I have tested with one USB
webcam and three virtual video devices (Fake Webcam, ManyCam, and
screen-capture-recorder).

I have attached a small patch that shows the first 12 bytes of
block_t's just before they are passed to es_out_Send. In this case the
capture source outputs bluish solid-color frames of R=0x2b, G=0x9b,
B=0xd8

When capturing in RGB24:
dshow debug: DShow format: RGB24
dshow debug: Block buffer: 0xd8 0x9b 0x2b 0xd8 0x9b 0x2b 0xd8 0x9b
0x2b 0xd8 0x9b 0x2b

When capturing in RGB32:
dshow debug: DShow format: RGB32
dshow debug: Block buffer: 0xd8 0x9b 0x2b 0xff 0xd8 0x9b 0x2b 0xff
0xd8 0x9b 0x2b 0xff

The color masks used in both cases are the same:

fmt.video.i_rmask = 0x00ff0000;
fmt.video.i_gmask = 0x0000ff00;
fmt.video.i_bmask = 0x000000ff;

However, in the case of RGB24 VLC renders colors incorrectly. With
RGB32 everything is fine.

So again this does not seem to be a DShow bug. Maybe the RGB24 color
masks are interpreted incorrectly in chroma.c?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dshow-rv24.patch
Type: application/octet-stream
Size: 1646 bytes
Desc: not available
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20170211/794290af/attachment.obj>