[vlc-devel] [PATCH] provide the display format from the demuxer to the vout to respect cropping

[Steve Lhomme]

On Mon, Mar 9, 2015 at 8:09 PM, Rémi Denis-Courmont <remi at remlab.net> wrote:
> Le lundi 09 mars 2015, 18:26:17 Steve Lhomme a écrit :
>> And I explained each time why there is information missing at the vout
>> level. I even explained in this thread which you may have missed.
>
> Or not? There are no questions that the format supplied to the video output
> needs to take the container crop parameters into account, and there never was.

I explained what I was trying to do and the bugs I was finding along
the way. The obvious solution doesn't work (green areas or bleeding).
So I tried to understand why, especially since user crop is working
fine. This patch is the first one of the series that I consider fully
working.

> *But* you have not explained why the video output would ever need to make
> distinction between the container and the codec crop parameters. And indeed,
> it makes absolutely no sense for the video output to distinguish those: there
> are just two different manners to convey the exact same information.

I think one of the reason it's not working is, as described earlier in
the thread, the way the vout is initialized and that certain values
must be set only once the vout is ready. The underlying code must be
dirty and is clearly marked as such. So rather that spending my time
finding all the things are done wrong, and considering parts of the
code and new to me, I preferred to go around the issue.

I also had hope that this could be fixed easily, since it was working
in 2.1. But the amount of changes is so huge that it would take
considerable time to find out where/when it broke. And even having a
patch from that information in the newer code might not even be
possible.

> In hindsight, the XVideo bleeding taught us that it would be nice to
> distinguish between cropping (valid data that the user does not want to show)
> and padding (alignment and buffer for optimization with no valid data). But the
> cost of implementing such distinction is large, and the benefits are seemingly
> limited. In any case, this is _not_ what this patch does.
>
>> " The reason not to do this in the decoder instead of the vout is that
>> the vout is hackish and sets up the texture and things like that
>> before applying the dimensions. We need to keep the original
>> dimensions at that stage.
>
> Oh, I am quite sure that I mentioned (at least on IRC), that initial cropping
> was probably not working in the video output. Nevertheless that bug lies
> inside the video output core and possibly some video output display plugins.

I went and tested a lot of options because "probably" and "possibly"
didn't help much in finding who is supposed to do what in what
conditions and what was wrong. The one thing I found working is the
user crop, that's why my solution is based on that thing already
working.

As I said to j-b, I'm fine with a codec based solution that would be
more optimized. But there are so many cases where it can fail, that a
global solution makes more sense, at first.

> There are no needs of, and no uses to, changing the video output external
> interface to address this particular problem.

Now that's one good reason to avoid doing it my way.

>> You proposed no working solution nor anything close to a useful hint.
>
> I spent a lot of my free time discussing this on IRC and on this list. Those
> accusations are utterly unfair.
>
> You have wasted enough of my free time and expanded enough of my motivation.
> This is intolerable, especially considering that you are ostensibly paid to do
> this.

I don't see what that has to do with the topic. But I don't want this
to turn into a personal flame war.

> The topic is closed as far as I am concerned. If you post something like this
> again, I will refer your case to the board for mailing list ban.

How can the topic be closed when the regression is still there unfixed
? We need to find a working solution to this issue. And that's why I
spent most of my time for a week to try to understand the inner
workings and limitations of the whole system and come up with a
working solution that I find elegant.

I'm also fed up with the subject. I'm fine updating this patch, but no
more for now. I don't want to spend another week fixing a dozen codec
and a dozen vout that were marked as hackishly working years ago.

Best regards,
Steve