[Android] Presentation

[Martin Storsjö]

On Mon, 4 Feb 2013, Jean-Baptiste Kempf wrote:

>> To do so, we have to make our OMX hardware codecs interoperable with VLC.
>
> Or fix our OMX wrapper :)
> To be honest, it is very possible that our OMX code is faulty.

Yes, this is pretty much the case for any not-yet tested codec mapping - 
it's too much to wish for to hope that blindly sending our extradata as 
CODECCONFIG and packets just untouched will work, unless it's been tested 
and seen to actually work on some existing device out there - otherwise it 
is just likely to break.

>> Issues we have seen for the time being are compatibility between demuxers & decoders & A/V sync issues.
>
> Could you detail those issues, please?

I've discussed with Hugues and Rafael off-list about the audio codecs that 
were mapped in 
http://git.videolan.org/?p=vlc.git;a=commitdiff;h=2b101ff4a0464f10b7748ca065dcbfd2b3173f08 
- currently the situation seems to be (on my Galaxy S3):

- mp3 works fine (both with the samsung decoders and with google's sw 
decoder), so I'm ok with keeping the mapping. MPGA (tested with an mp2 
file) works fine with Samsung's decoder, but not with google's, so I'd say 
we should remove that mapping as we apparently can't expect any random 
decoder that has the role audio_decoder.mp3 to decode the older forms of 
mpeg audio.

- Using vorbis currently via OMX is wrong. The only OMX vorbis decoder I 
know of is OMX.google.vorbis.decoder, and with that one our code requires 
a few modifications to make it work, see 
https://github.com/mstorsjo/vlc/commit/omx-vorbis. (Additionally, 
OMX.google.vorbis.decoder is slightly broken, which is why I need 
https://github.com/mstorsjo/vlc/commit/cd5d64f57f35c53d7a236001d9f0a9ade48e6b30 
for testing, fix sent upstream in 
https://android-review.googlesource.com/50871.) While the code does work 
with these changes, the bitstream format sent seems pretty hacky/not 
standardized, so I'd be a bit reluctant to keep it at all, especially 
since there's not all that much value for VLC to keep it (we've got our 
perfectly good sw vorbis decoder anyway, and the performance of it isn't 
an issue normally). Since we ignore OMX codecs named OMX.google normally 
we wouldn't use it even if we kept the mapping.

Also, for vorbis/ogg, stagefright already does have a working ogg demuxer, 
so if one is creating an OMX vorbis decoder, make sure it works with the 
stagefright setup - that one is what vendors would use it with anyway, 
what VLC does is not a reference but mostly a lucky conincidence if it 
happens to work, and what we do currently is wrong.

- WMA currently does not work with the samsung decoders (so WMA playback 
which worked just fine prior to adding these omx codec mappings suddenly 
is broken on my device, since it tries to use omx codecs in an untested 
way). Hugues did point me in the right direction though, and with his 
hints I was able to make it work for certain simple wma1/2 files, but I've 
also got a bunch of wma2 streams that still does not work (but works fine 
with the avcodec sw decoder, obviously), and wma lossless and wma pro 
doesn't still work with the samsung decoder with my patches. So unless I 
can make this work reliably at least on my device (and ideally on any 
other OMX wma decoder), I'd say we should disable this mapping.


So, to summarize, the situation about the newly added codec mappings in 
the commit discussed is:
- mp3, keep
- mpga, disable IMO
- vorbis, either keep or disable, depending if we think the bitstream 
format seems like something google is going to keep as such, as de-facto 
standard
- wma, disable unless I can make it work for all streams soon

// Martin