[vlc-devel] [PATCH] When the demuxer sets dimensions use them rather than the codec values

[Steve Lhomme]

On Fri, Mar 6, 2015 at 11:38 AM, Rémi Denis-Courmont <remi at remlab.net> wrote:
> Le 2015-03-06 13:26, Steve Lhomme a écrit :
>>
>> On Fri, Mar 6, 2015 at 11:11 AM, Rémi Denis-Courmont <remi at remlab.net>
>> wrote:
>>>
>>> Le 2015-03-06 12:25, Steve Lhomme a écrit :
>>>>
>>>>
>>>> Your first assumption is false. I don't know about MP4 but in Matroska
>>>> the coordinate mapping is defined. It applies to the output format of
>>>> the codec.
>>>
>>>
>>>
>>> The output format of the decoder depends on the decoder implementation.
>>> If
>>> the Matroska specification depends on that, then the Matroska
>>> specification
>>> is broken. And my assumption still holds.
>>
>>
>>
>> No. That has nothing to do with implementation.
>
>
> Then the patch is wrong, because of exactly what I was writing earlier: only
> the decoder knows the mapping, not the core.


As said in the patch, it creates bleeding with some vout. But since
you mentioned that maybe the container should overwrite the codec
values when they are present, I tried.

I'm investigating why using --crop 164+86+180+90 on the sample works,
but applying this patch creates bleeding. It's down to a confusion
between the use of i_visible_width/height that is used to setup the
output planes here
http://git.videolan.org/?p=vlc.git;a=blob;f=modules/video_output/msw/direct3d9.c;h=f333471d5fd3cadd176fa869fcaaaf9d7dcff0b8;hb=HEAD#l1000

That part must be unaltered from the cropping data as that's what. And
it relies on i_visible_width/height and i_width/height. Even when the
command-line crop is involved, that part gets 0,0 for the x,y offsets.
If anything varies from what the decoder outputs, we get green/garbage
pixels.
http://git.videolan.org/?p=vlc.git;a=blob;f=src/misc/picture.c;h=6aa5b056a9105d491f66a1d026d3df2b6b627043;hb=HEAD#l211

So to achieve proper "format" cropping we need to keep the
video_format_t given by the decoder and pass the extra info one way or
another.

It's not impossible though as it's already working when passing the
crop data in the command-line, even top-left values. It works with
Direct3D9, DirectDraw, OpenGL, Windows GDI but not Direct2D.