[Android] Faster Video rendering

[Martin Storsjö]

On Thu, 28 Feb 2013, Jean-Baptiste Kempf wrote:

> Hello,
>
> After discussing with People from Google/Android, it would seem that the
> recommended ways, for us, to output video is:
> - ANativeWindow on 2.x (instead of SurfaceFlinger)

This probably won't make rendering faster (since it's basically exactly 
the same API as we're using right now, but using a public ABI/API for it), 
but the conversion should be pretty straightforward. The drawback is that 
it's only available on 2.3 though. So we'd still need to keep the current 
hacks for 2.1/2.2, and then it doesn't add much to add an extra codepath 
just for 2.3/3.x.

But just for the sake of it, I've got an experimental patchset that 
converts the current androidsurface vout to use ANativeWindow, at 
https://github.com/mstorsjo/vlc/commits/nativewindow and 
https://github.com/mstorsjo/vlc-android/commits/nativewindow. On Nexus 7 
it seems to work just fine, but on Galaxy S3 the video gets tearing, 
looking like the buffer is getting displayed while it is filled. If 
someone wants to look into it, go ahead.

> - SurfaceTexture on 4.x
>
> I doubt this is urgent, notably the last part, because it is a lot of
> work, but it would be nice...

Did they have any suggestions on how to use SurfaceTexture? As far as I 
can see, this is an opaque class that we can't fill ourselves, we can only 
use it as target for e.g. camera preview and a few other android classes, 
but we can't pass raw data to it ourselves.

Or if you mean rendering directly into it from the MediaCodec decoder - 
yes, that's obviously faster, but requires quite a bit of changes (as 
discussed earlier).

On this topic, I've got a very much experimental patchset that tries to do 
direct rendering with MediaCodec, see 
https://github.com/mstorsjo/vlc/commits/mediacodec-render and 
https://github.com/mstorsjo/vlc-android/commits/mediacodec-render. There's 
a number of issues to sort out, if someone wants to look into it, please 
feel free.

This should be able to perform almost as well as the system media player 
and give 1080p playback on devices where we can't do it currently (but 
where the system media player can do it), but obviously only works on 
devices running 4.1.

// Martin