[Android] [vlc-android] H264 1080p perf issue

[Hugues FRUCHET]

Hi Martin,

I don't have solution for stagefright utility inside an apk to assess perf, but what I understand from the result we have up to now is that OMX decoder performance needs to be largely above the framerate required for the stream, I suppose due to the 2 entire frame copy + YUV->RGB frame conversion.
Gallery app uses OMX AndroidNativeBuffer extensions, aiming to achieve 0-copy by giving to decoder the graphic buffer details where to decode in. 
Then buffers are sent to composition stage which get access to graphic buffer details in hwcomposer HAL, so vendors can achieve 0-copy by matching their OMX decoder buffers implementation with their hw compositor buffer implementation (gralloc).
So I was wondering if those extensions are exposed or not through IOMX, and it seems they are.
I have found a player, MXPlayer which uses those IOMX extensions to address the hardware OMX decoder, and results are very close to what I obtain with Gallery (no more performance problem, and I can do h264 1080p 30fps).
If you want I can send to you the OMX logs with this player.

I have not checked the code and I don't know how they got the graphic buffer handle they give in IOMX->UseBuffer, I suppose we can get it from Android Surface jni stuff.

Is this an option you have explored ? or are there some reasons to keep buffer output mode (smarter YUV->RGB conversion for example, ...) ?


BR.
-----Original Message-----
From: Martin Storsjö [mailto:martin at martin.st]
Sent: mardi 19 février 2013 14:20
To: Hugues FRUCHET
Cc: android at videolan.org; Rafaël Carré
Subject: RE: [Android] [vlc-android] H264 1080p perf issue

On Tue, 19 Feb 2013, Hugues FRUCHET wrote:

> Here are the results on my side:
>
> * With Rafael transcode patch:
> $> adb shell am start -n 
> org.videolan.vlc/.gui.video.VideoPlayerActivity -a 
> android.intent.action.VIEW -d file:///mnt/sdcard/bbb_1080p.mp4 Played till end of video $> adb logcat
> W/VLC     ( 1168): [0x1e0b6e4]: stream_out_transcode stream out Decoding took 50906 ms
>
> 60s of video at 24fps => 1440 frames, 1440/50.906 = 28.28fps
>
> So perf is lower than on your side.
>
>
> * With Stagefright
> $ adb shell stagefright /mnt/sdcard/bbb_1080p.mp4
> thumbnailTime: 10041666 us (10.04 secs) AVC video profile 77 and level 
> 41 
> ......................................................................
> ....................$
> avg. 42.44 fps
> avg. time to decode one buffer 22866.01 usecs decoded a total of 1439 
> frame(s).
>
> But with Stagefright unit test, perf is not so bad.
>
> Would be nice that we got same measurement on your side, in order to known if problem is located on decoder itself or due to how VLC uses the decoder.
>
> What we could try is to push the stagefright utility on your device, which version of Android do you run ?

My device runs Android 4.1.2, but even if you have a binary that fits (I could build one myself pretty easily as well), I can't install and run it easily since it's not a rooted device. If you can easily package in this binary into a normal apk that allows running tests via it, it could be possible though... I don't have time to do that myself right now though.

// Martin