[Android] How to debug A/V sync issues when dealing with OMX hardware decoders ?

[Hugues FRUCHET]

> Le 04/02/2013 11:24, Hugues FRUCHET a écrit :

> > Hi all,

> >

> > First of all, I have searched on VLC bug tracker (https://trac.videolan.org/vlc) &  VLC forum (http://forum.videolan.org/) but > couldn't get much infos on how to debug such issues (mainly found some reference to audio output opensles bug fixes).

> > I'm using VLC apk 0.0.8 with Android ICS and OMX hardware codec interface.

> >

> > When adding support of some audio & video OMX hardware decoders in VLC, I faced 2 majors problems:

> >

> > 1)      Audio drops

> > Example of warning message:

> > W/VLC     ( 1173): [0x14c72cc]: main audio output PTS is out of range (479247), dropping buffer

> > From src/audio_output/input.c

> >

> > I wanted to know what are your advices to debug this type of problem ?

> > For example, I have observed with MP3, that if decoder bufferisation is too big, decoded samples are claimed as too late, but it's > only a problem of latency, seems that reference time taken by VLC is not the first sample received but the first sample sent.

> > Reading code, I cannot really understand how it works.

> > By reducing latency of decoder, I have fixed this exact problem.

> > Is there a patch or an option to set in order  to disable the checks on lateness and let audio be rendered without any check ?



> No there's no such patch.

> I think you should increase the audio packets PTS if you're sending them

> to the output a while after decoding.

[HFR] I don't see how to do that, I don't know what is the ref time on the VLC point of view (I cannot be sure of the system time used by VLC). As far as I have seen in gstreamer for ex., the pre-roll phase is here to do this (get rid of initial config/setup phase of decoder).



> Why do you need high latency in your decoder?

[HFR] I don't need audio latency, it's something I had to deal with, based on current implementation.

To summarize, I suppose that the decoders are assumed by VLC to be "0-delay" (or few delay), it's why I have reduced to a minimum

The latency, but this is not needed for Stagefright, so I am wondering why such difference.





> > 2)      A/V sync issues with video late

> > With some files, video is rendered completely out of sync regarding audio, video is late by some seconds and some video drops are observed.

> > I don't have fresh log to give on this exact issue I have faced some weeks ago, but I'll restart this activity this week, and I want to restart with a better basis

> > regarding VLC debug tips.

> > *Have you some patch or options to set in order to disable audio and let video be rendered as fast as it can ? This would allow to test the pure performance of video chain, including OMX decoder, YUV=>RGB conversion & display. With Android Gallery apk, I don't face perf issue and A/V is in sync



[HFR] I have redone measurements, problem I have is that FillBufferDone are there in average every 56 ms while 40ms expected (25fps H264 movie)



> I do have a patch to test decoder performace, but that completely

> bypasses display/rendering. It's not very convenient as it disables

> audio/video output completely (it's not configurable at runtime so you

> have to use a custom patch).

[HFR] I think it could be usefull for me, my favorite Android utility is "Stagefright" in cmd line (adb shell stagefright --help), a utility which replaces the stagefright player by a limited player with no output and "decode in buffer as fast as possible" strategy. Then a perf figure is displayed allowing to know the unit perf of decoder, getting rid of video output.

Would be nice to have the same with VLC omx integration so I could know if my perf problem comes from OMX buffers handling for ex.





> I might be able to adapt it to add YUV->RGB conversion, I need to check.

[HFR] I have tested by disabling the YUV to RGB conversion in VLC (comment of nv12_rgb_neon in NV12_RGBA in yuv_rgb.c.

Only 6ms gained (50 ms instead 56ms for FillBufferDone period), so it's not the problem.



> BTW did you try the YV12 rendering option in advanced preferences? (It

> is broken on several devices but might work for you)

[HFR]I don't have "Advanced preferences" in preferences, I have "Advanced debugging" with SetLocale but I don't know how to use it.

Anyway seems not YUV->RGB conversion issue.



> > *For YUV=>RGB conversion, how can I know the performance (duration) ? is the neon optim enabled or not (trace ?)



> Performance is not checked atm but we could add some counters.

> NEON should be enabled, you can see it in logcat : look for "yuv_rgb_neon".

[HFR]OK



> > *When reading code, I cannot found the A/V sync mechanism I usually encounter in Stagefright/AwesomePlayer.cpp (or other players I have encountered in the past) which consist to take the audio clock (system clock if no audio) as reference and render the decoded video frames when the audio clock has elapsed based on the video frames timestamp. Have you some doc which explain this ?



> No there are no docs, the video output code was written by fenrir and

> he's not available to explain it.



> Video logic is all in src/video_output, and audio rendering

> synchronization logic is performed by audio_output_t->time_get

[HFR]OK, I'll have a look to this

[HFR]If I want to disable completely audio side of playback (skip audio samples at demuxing side for ex.), what is your recommendation ?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/private/android/attachments/20130205/7a4bb12c/attachment-0001.html>