[vlc-devel] [PATCH 0/2] Filters with chroma conversion

[Steve Lhomme]

On Fri, May 5, 2017 at 10:30 PM, Rémi Denis-Courmont <remi at remlab.net> wrote:
> Le perjantaina 5. toukokuuta 2017, 22.06.28 EEST Steve Lhomme a écrit :
>> On Fri, May 5, 2017 at 6:46 PM, Rémi Denis-Courmont <remi at remlab.net> wrote:
>> > Le perjantaina 5. toukokuuta 2017, 10.14.01 EEST Steve Lhomme a écrit :
>> >> Right now we cannot use hardware decoding with CPU filters. The decoder
>> >> outputs an opaque chroma that CPU filters can't deal with.
>> >
>> > You can´t do it because filters deliberately reject GPU input formats, for
>> > performance reason.
>>
>> What's the performance hit of doing this ?
>
> The same reason that we don´t enable video decoding without direct rendering.
> Copying from GPU memory to CPU memory is slow, usually no faster than decoding
> on CPU, and sometimes much slower.

I did some performance tests. For 1080p decoding + filter there is no
frame dropped in software decoding or hardware decoding. The CPU usage
is way down with the hardware decoding, although the CPU double copy
makes the display thread very busy. It's still better battery wise.
For 4K 60fps HEVC decoding it's trickier. With an integrated Intel GPU
there's no frame dropped but there are about 20% frames dropped with a
discrete GPU. That probably has to do with the fact the memory is the
same chip(s) with an integrated GPU but not with a discrete GPU. But
with that source software decoding is not fast enough anyway, let
alone with an extra filter, so there's no real difference.

In all cases we save on battery usage and we have all the possible
filters available, not just the ones that can be done by the GPU.

>> > The core already supports converting.
>
>> Can you point me out the code that does conversion in the core ?
>> That's precisely what I've been looking for and had to create to make
>> this work.
>
> Frankly, it´s hard to miss the filter_chain_AppendConverter() calls, or its
> older incarnation, if you look at the video output filter management code.

There is the decoder+filter/vout adapt chain in video_output/display.c
but it's not used because the filter is never added to the chain in
the first place. Here is an example of trying to add a filter when
using DXVA2 decoding:

core video output debug: Adding 'adjust' as interactive
core filter debug: looking for video filter module matching "adjust":
45 candidates
adjust filter debug: Unsupported input chroma (DXA9)

All we need is an interim step that adds a middle man (likely I420)
before adding filters. Then they can all work in I420 and the chroma
adapter in video_output/display.c will kick in. But it can't happen
without this middle man.

Another possibility I found while looking at encoders is that filters
could set the chroma they expect on input. Many encoders already set
the codec/chroma they want on input. When adding a filter in the
chain, care will have to be taken so that input and output matches.

> --
> 雷米‧德尼-库尔蒙
> https://www.remlab.net/
>
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel