[vlc-devel] GSoC 2018 Idea `libplacebo Improvement'

[Niklas Haas]

Hello,

On Tue, Mar 20, 2018, at 16:15, zhzhzoo zhang wrote:
> Hello everybody,
> 
> I am a first year graduate from Peking University major in math.
> I started to take part in algorithm competitions at grade 7, so I
> am very well trained in algorithms and data structures and of
> course coding.> I am especially interested in the idea `libplacebo Integration &
> Improvement', in the description it seems integration and improvement
> would be assigned to two students, and I want to do the improvement
> part.

To clarify: The separation between students was never so much intended
as an "integration and improvement"-type separation. There are many
potential avenues for improvement, some of which I will also be
attempting during my GSoC time. And if you are interested in providing
Metal support, it would also be your job to add the Metal-based
integration for VLC. (Which should be easy to provide if we abstract out
the libplacebo code into a common vout_placebo)

> (possibly because I want to learn some graphics.) I successfully
> compiled libplacebo, and looked through its issues. There says its
> renderer needs more tests while other parts are fairly covered. So is
> this the reason why libplacebo isn't production-ready?

While more test cases would be nice, lack of test cases is not the main
issue with libplacebo. The main issue is that some important features
are still missing. I will most likely be shifting my attention towards
those after finishing up the VLC integration, including:

- color management and ICC/3DLUT support (partially done)
- frame (redraw) caching (untouched)
- expanding on that, frame mixing and interpolation (untouched and
  difficult)
- (possible) custom shader support (nothing planned)

In general, I want libplacebo to have feature parity with mpv's vo_gpu.

> I want to implement Metal support because I use a mac and hope my
> computer can be supported. Apart from these, what else features for
> libplacebo are needed? (i.e. what does the word `missing features' on
> the idea page refer to?) Thank you very much!

Metal support would be a possibility for sure, and that would definitely
be a project big enough for a student to work on. As tguillem mentioned
however, MoltenVK may be a more realistic option for getting support
onto macOS platforms.

For Metal specifically, the biggest problem is most likely going to be
cross-compiling from GLSL to MSL, which requires SPIRV-Cross and a
custom C++ wrapper for it at the very minimum. (I believe that's what
MoltenVK also uses internally)

For missing features, see this issue tracker:
https://github.com/haasn/libplacebo/issues/

On Tue, 20 Mar 2018 21:03:48 +0100, Thomas Guillem <thomas at gllm.fr> wrote:
> Hello,
> 
> Niklas, the creator of libplacebo will know how to respond.
> 
> As far as Metal is concerned. We need to have a discussion about using
> or not MoltenVk since the Vulkan support is quite advanced in
> libplacebo now.

> The main goal of using a new vout for ios/macOS is to allow direct
> rendering with SW decoders. For that you need to be able to permanently
> map a GPU buffer to the CPU and give this buffer to the decoder
> (avcodec).

> You'll need to check if it's possible with MoltenVk or Metal.
> For information, we can't do it with OpenGL since this feature was added
> in OpenGL 4.4 and Apple stopped updating the OpenGL  in macOS/iOS (that
> is stuck to OpenGL 3 or so).

It's possible with Vulkan for sure, based on the fact that you're seeing
something on-screen at all with MoltenVK. Mapping memory is a core part
of vulkan operation and also libplacebo code. So by proxy, it's possible
with Metal.