[vlc-devel] Video processing APIs

[Andre-John Mas]

On 18-May-2009, at 18:03, Jean-Baptiste Kempf wrote:

> On Mon, May 18, 2009 at 11:27:02PM +0300, Rémi Denis-Courmont wrote :
>> Slightly up the stack, it is just as messy with those challengers:
>> - VA-API (Intel)
>> - VDPAU (NVidia)
>> - Xvideo BA (AMD/ATI)
>> - DirectX VA (Microsoft)
>> - OpenMAX integration layer
>> - I wonder what Apple is doing?
>> And of course, we still have Xvideo Motion Compensation on older  
>> hardware.
>
> Windows states of affair seems easy: DxVA and almost everyone  
> implements
> it.
>
> Linux is a bit more complex, but VAAPI seems to be nice, because there
> is already a working VDPAU backend and a half-working XvBA backend to
> it.
>
> Apple seems to speak about OpenCL all the time, but I haven't seen a
> video decoding API (except QT X, maybe) using GPU. And I doubt that  
> they
> will release one since they don't think anything else than QT should
> decode video. And they are so open lately...
>
>
> CuDA and OpenCL are more for video filtering, I guess...


Although we do risk some fragmentation, it is certainly best to have  
an implementation that uses one of the APIs, than wait for ever. I  
also agree that voting with code is better than endless 'comity' style  
discussions. Better have something that we can use, than a pipe dream.  
I certainly look forward to the results of Etienne's summer of code.

Looking at OpenMax, this appears to be higher level then OpenCL, so it  
is quite possible we will see OpenMax based codecs that will take  
advantage of OpenCL for its low level work. I would imagine that with  
OpenMax we would see the implementation of certain codecs move out of  
the ffmpeg/VLC space and into something that is accessible more widely  
at an OS level?

ref: http://www.khronos.org/openmax/

Reading further I see that VA-API is at the same level as OpenMax, so  
likewise it can base itself on whatever is available, whether it be  
OpenCL or DirectX Compute. I assumed, incorrectly, that VA-API was at  
the same level as OpenCL.

ref: http://en.wikipedia.org/wiki/VaAPI

As to OpenCL, we will probably see more happening once Apple has  
released Snow Leopard, and from what I have read there is certainly a  
want to push this as an industry standard. I suppose the deciding  
factors will be the terms of licensing and the ease of implementation.

OpenCL is more of a number crunching API, that can take advantage of  
specialized processors, such as what is found in the form of GPUs.

ref: http://www.khronos.org/opencl/

 From what I have gathered Nvidia has already shown interest to  
migrate their PhysX from CUDA to OpenCL, and they have even indicated  
they are interested in making it on ATI cards - I would guess this  
plays into making PhysX more interesting to potential licensees and  
would also help reduce the cost of code maintenance.

ref: http://www.bit-tech.net/news/hardware/2009/03/27/nvidia-considers-porting-physx-to-opencl/1
ref: http://www.xbitlabs.com/news/video/display/20090420170635_Nvidia_Enables_Non_Proprietary_GPGPU_by_Releasing_OpenCL_Drivers_to_Developers.html

André-John