[Android] A word on release and audio

[Ludovic Fauvet]

Hey,

On Thu, Feb 14, 2013 at 7:00 PM, Rafaël Carré <funman at videolan.org> wrote:
> Hello,
>
> I think we should make a new release as soon as possible.

Yes, please.

> Since the opensles is the only module handling audio/video
> synchronization, I think it should be the default.

Obviously this *must* be new default.

> It is certainly not perfect as noticed by j-b and etix last month, since
> *some* samples show lipsync issue.
>
>
> Unfortunately I have not enough material to make a fix and I think
> releasing with that module as default would give precious user feedback.

AFAICT I've seen the bug only on one device, the Archos 97 carbon we
have in the Paris office.

> That last patch I tried seems to change the behaviour of opensl module
> but unfortunately I have no idea why (see the end of the mail for an
> explanation).

No idea either, but it solves the issue for this specific tablet and sample.

> IMO releasing now would have 2 benefits:
> - testing 2 months of android.git commits that you probably all expect.
> - give me useful feedback about opensl:
>
> - if users say it's worse, i'll accept the statistical evidence and dig
> the code to find what i did wrong
> - if users say it's better, i'll feel confident that i did something
> useful and will look to further improve it
> - if users say it's the same, i'll feel good about not breaking things
> too much.

Sounds good but with or without the buffer length to 10 ms?
In theory, could it break something to move from 4 to 10 ms?

> -----------------------------------------------------------------------
>
> Now j-b and etix will think I distrust them for asking user feedback
> instead of relying on their own feedback, but the truth is more nuanced.

Not really but we have evidence that there is a problem somewhere.

> There is a problem, I acknowledged it already as soon as you made that
> report. I certainly trust what you say when you told me you had a
> lipsync issue.
>
> Which one?
> I don't know and I am sorry to tell you your testing didn't make the
> issue more clear.

In my opinion it only affects few hardware but not all. For example,
my GNex and N7 are behaving correctly.

> I will be honest here, I also had some concerns with your report:
>
> - I had to ask for a sample, while I expected the sample to be provided
> with the report (we need the exact same sample to reproduce the exact
> same bug)

You should have the TBBT sample already, isn't it?

> http://www.catb.org/esr/faqs/smart-questions.html#beprecise
> "If at all possible, provide a way to reproduce the problem in a
> controlled environment."
>
> IMO, using the exact same file sample is key to "controlled environment"
> and I expected you had the same interpretation than me on this one.

That would work if it affects all hardware, which is clearly not the case here.

> - After complaining that I needed more samples I was promised other ones.
> I'm still waiting for them today despite claims that "it's broken for a
> lot of files".

I've never told you that "it's broken for a lot of files" and I have
no evidence of that.

> - The issue that I was described is not the one that I could finally
> reproduce with the help of etix at FOSDEM (sync appearing only at the
> end of movie versus appearing immediately at the start).
> If we don't have the same symptoms I can not be sure we are experiencing
> the same bug.
> Now they *could* be the same bug but before I confirm it.

The only bug I'm aware of, appears after a long time > 15 minutes of playback.

> As the only developer working on the audio code I feel entitled to
> labeling the bug reports.
> If you ever closed a bug report on trac as incomplete or can't
> reproduce, I guess you will sympathize with me.
>
> Also I don't have a problem being alone working on this, I know manpower
> isn't an infinite ressource.
>
> -----------------------------------------------------------------------
>
> Maybe it is my fault for not asking precisely what I wanted to be tested
> (like different sampling rates on the same device, same video codec,
> same container).

Every time you asked me to test something, I did it. And If you want
me to try other samples I will.

> I am also very aware that you are very busy at the moment, but that does
> not make this report more useful to me.
>
> A lot of work is obviously needed to identify and fix this problem since
> all of us spent hours already on it and the bug is still here.
>
>
> Now I still want this to be fixed, and I still can't do it without your
> testing.
>
>
> Here is what I suggest:
> - documenting tests on a trac ticket so we can find them later
> - uploading each and every sample tested to a common location so other
> people can reproduce the problem in the same "controlled environment".

As I told you before in this email, reproducing the bug can be very
hard since not all devices are affected.

> - expliciting the devices tested. I don't remember the exact model or
> android version running on your devices (I think now you give me this
> info but I complained about it when you first reported it though).

I will. Open the ticket, I will put all informations I find useful on it.
Just note that since I'm going to the US for a week I won't be able to
work on this issue during that time but I'll do as much as possible
before I leave.

> - complete mediainfo output on these files or selected essential
> information (sampling rate, codec, channels, frame size)
> - logcat | grep VLC - output, with the msg_Dbg uncommented in
> opensles_android.c (although we should rather uncomment it in git since
> we still have issues).

OK.

> -----------------------------------------------------------------------
>
> Also I was not explicit about why I think the last change helps, let me
> fix that.
>
> OPENSLES_BUFLEN is currently 4, I changed it to 10.
>
> This define is used in 2 places:
>
>     /* XXX: rounding shouldn't affect us at normal sampling rate */
>     sys->samples_per_buf = OPENSLES_BUFLEN * fmt->i_rate / 1000;
>
> We calculate the number of samples to be put in each opensl es buffer
> (of which we have 255 available).
>
> For 48kHz we have:
> samples_per_buf = 4 * 48000 / 1000 = 192 samples
> 192 samples = 192 * 1000 / 48000 = 4 milliseconds
>
> For 44.1kHz we have:
> samples_per_buf = 4 * 44100 / 1000 = 176 samples
> 176 samples = 176 * 1000 / 44100 = 3.9909297052 milliseconds
>
> If we add up our 255 buffers, that makes 1017.6870748260 milliseconds,
> instead of our expected 4 * 255 = 1020 milliseconds
>
> Thus it makes a difference of 2.3129251740 milliseconds in drift
> calculation, way below the recommanded 60 or 80 milliseconds advised by
> the DVB consortium (under these values we consider there is no lipsync
> issue), thus the comment "rounding shouldn't affect us at normal
> sampling rate" (where I meant 'normal' as in 44.1 or 48kHz).
>
> It is even below the 4ms accuracy that we have.
>
>
> The second place is the drift calculation itself:
>
>     *drift = (CLOCK_FREQ * OPENSLES_BUFLEN * st.count / 1000)
>         + sys->samples * CLOCK_FREQ / sys->rate;
>
> (st.count is the number of busy buffers, between 0 and 255).
>
>
> Bumping the value to 10 effectively makes 44.1kHz round nicely to 10ms =
> 441 samples exactly.
>
> But it doesn't explain why a 2.3ms difference makes the lipsync go away
> while our precision is limited to 4ms.
>
> It also doesn't explain why it makes lipsync go away on the big bang
> theory sample (the only one we have in common except maybe billy elliott
> that I uploaded and that I think only etix tested), because this one has
> 48kHz audio which has no rounding error with 4ms or 10ms BUFLEN.
>
> -----------------------------------------------------------------------
>
> I hope this will clear any misunderstanding and remove any feeling that
> I'm responsible for the lack of interest in the android work (I do not
> see this lack of interest myself anyway).

I was a bit upset yesterday so please accept my apologies, I didn't
want to be disrespectful with you.
That being said, I see absolutely no reason to hold a release because
of the introduction of a new audio output since we're still in beta
and people are fully aware of that.

Kind regards,
--
Ludovic Fauvet