[Android] [vlc-android] H264 1080p perf issue

[Martin Storsjö]

Hi Hugues,

Let's move the discussion back onto the mailing list so that others can 
benefit from the discussion as well.

On Mon, 18 Feb 2013, Hugues FRUCHET wrote:

> Here is the try with patch you suggested (comment of decoder_new_picture and copyomx):
> Video stuck after 7 frames, and here are the delta between FillThisBuffer (activity of video decoder thread tid 1217):
>
> 09-01 00:05:18.419  1186  1217 D VLC     : [0x1ab601c]: iomx decoder FillThisBuffer 0x1abc160, 0x18f74000
> 09-01 00:05:18.440  1186  1217 D VLC     : [0x1ab601c]: iomx decoder FillThisBuffer 0x1aba810, 0x18c76000
> 20ms
> 09-01 00:05:18.449  1186  1217 D VLC     : [0x1ab601c]: iomx decoder FillThisBuffer 0x1abc380, 0x19570000
> 10ms
> 09-01 00:05:18.479  1186  1217 D VLC     : [0x1ab601c]: iomx decoder FillThisBuffer 0x1abbfd0, 0x19272000
> 30ms
> 09-01 00:05:18.509  1186  1217 D VLC     : [0x1ab601c]: iomx decoder FillThisBuffer 0x1abc160, 0x18f74000
> 30ms
> 09-01 00:05:18.539  1186  1217 D VLC     : [0x1ab601c]: iomx decoder FillThisBuffer 0x1aba810, 0x18c76000
> 30ms
> 09-01 00:05:18.620  1186  1217 D VLC     : [0x1ab601c]: iomx decoder FillThisBuffer 0x1abc380, 0x19570000
> 81ms
>
> I don't know really how to interpret this.

I take it this means that you get 6 frames in 201 ms, which is almost 30 
fps on average. This is a noticable improvement compared to the full 
rendering process, right?

> What I can say is that using Stagefright command line utility to assess decoder performances in NV12 with buffer copy in output, I obtain about 38fps, so far more than the 25fps needed for this file:
> $ adb shell stagefright -m100 /mnt/sdcard/Irma-I_know.mp4
> thumbnailTime: 8000000 us (8.00 secs)
> AVC video profile 100 and level 40
> .......$
> avg. 38.26 fps
> avg. time to decode one buffer 20694.87 usecs
> decoded a total of 100 frame(s).
>
> Find attached "trace_irma_stagefright_cmdline_trace.txt" for the OMX trace using this command line.

Right, so that's a bit better, but the difference is not all that huge.

Please do note that VLC will not try to play things back faster than 
realtime by default - or does the patch from Rafaël make it decode as fast 
as possible? - After trying to apply the changes from the previous mail 
and trying to play back a video, it still plays back at normal speed.

So, no matter how fast decoding you have, you will not get less time than 
~33 ms between each call to OMX_FillThisBuffer on average (for 30 fps 
video).

> In order to progress on this issue, could we try to compare the trace you obtain with a Galaxy S3 and mine with the same video only file ?
> The video only file (1080p h264) is 23MB, I don't know how to share it, but perhaps have you a content h264 1080p that I could download ?

One 1080p H264 sample I can share is 
http://albin.abo.fi/~mstorsjo/bbb_1080p.mp4.

With this sample (including rendering of the samples, I haven't commented 
out the decoder_NewPicture and CopyOmx* calls), I get the following times 
between each OMX_FillThisBuffer call:

D/VLC     (26367): [0x69a33c4c]: iomx decoder FillThisBuffer 0x5f51d8c0, 0x63850000 20 ms
D/VLC     (26367): [0x69a33c4c]: iomx decoder FillThisBuffer 0x69a367b8, 0x63552000 94 ms
D/VLC     (26367): [0x69a33c4c]: iomx decoder FillThisBuffer 0x5f51d8c0, 0x63850000 10 ms
D/VLC     (26367): [0x69a33c4c]: iomx decoder FillThisBuffer 0x69a367b8, 0x63552000 17 ms
D/VLC     (26367): [0x69a33c4c]: iomx decoder FillThisBuffer 0x5f51d8c0, 0x63850000 60 ms
D/VLC     (26367): [0x69a33c4c]: iomx decoder FillThisBuffer 0x69a367b8, 0x63552000 36 ms

So at this segment it's 237 ms for 6 frames, around 39 ms on average, or 
25 fps.

At another section in the same file, the average is 36 ms, or 27 fps. The 
file itself is 24 fps, so it is played back more than fast enough, and 
since VLC plays it back at realtime speed, it will never become faster 
than this. So this is not a meaningful benchmark, but this phone decodes 
1080p content "more than fast enough". To get to know exactly how fast, 
via VLC, we'd need to hack VLC more to not throttle it to realtime.

// Martin