[vlc-devel] [RFC 0/8] Ambisonics channels support

[Adrien Maglo]

Hello,


These RFC set of patches aims to open the discussion about how we should handle the Ambisonics channels inside VLC.

Ambisonics is a full sphere sound surround format where each audio channel carries the directional information of the sound rather than the output for a particular speaker. An Ambisonics stream can be rendered with different speaker setup at the condition that the minimum number of speakers is equal to the number of Ambisonics channels. The number of Ambisonics channels of a stream can be calculted with its order with a simple formula: nb_channels = (order + 1)^2
This means that an order 1 stream has 4 channels, and order 2 stream has 9 channels and an order 3 stream has 16 channels and so on. Consequently, as you can note, we quickly reach the hard coded limit of VLC on the number of audio channels (AOUT_CHAN_MAX=9). It will probably need to be increased.

This Ambisonics B-format has been recently chosen by Google to support 3D sound on Youtube: https://github.com/google/spatial-media/blob/master/docs/spatial-audio-rfc.md
Current Ambisonics Youtube streams are of order 1 but this may increase to order 3 in the near future.

To support Ambisonics inside VLC, we would need to differentiate the handling of traditional physical channels and Ambisonics channels. Indeed, before being rendered, Ambisonics streams must be decoded for a predefined speaker setup.
The first patches proposes a *proof of concept* and *unperfect* solution to distinguish them. It implies a small change in all channel mixer, audio output modules and probably more... I would be glad to get suggestions for a better design.

A next set of patches will add an Ambisonics decoder based on an external library. It is an audio converter module that decodes the Ambisonics channels into physical channels for a chosen speaker layout.

Best regards,


-- 
MagSoft

Adrien Maglo (8):
  ES: add a property to audio_format_t for Ambisonics support
  ES: similar audio format have the same channel_type and
    i_ambisonics_order
  aout: check and keep the channel_type and the ambisonic order
  aout: display "Ambisonics" and the order for the channel format
  alsa pulse: force the Ambisonics to binaural conversion of the input
    (incomplete commit)
  codec: keep the input channel_type (incomplete commit)
  channel_mixer: refuse to convert ambisonics tracks (incomplete
    commit?)
  mp4: read and save ambisonics metadata from the Google specification

 include/vlc_aout.h                           |   1 +
 include/vlc_es.h                             |  13 ++
 modules/audio_filter/channel_mixer/simple.c  |   3 +
 modules/audio_filter/channel_mixer/trivial.c |   3 +
 modules/audio_output/alsa.c                  |   8 +
 modules/audio_output/pulse.c                 |   8 +
 modules/codec/araw.c                         |   1 +
 modules/codec/avcodec/audio.c                |   1 +
 modules/codec/faad.c                         |   1 +
 modules/demux/mp4/essetup.c                  |   5 +
 modules/demux/mp4/libmp4.c                   |  27 ++++
 modules/demux/mp4/libmp4.h                   |  12 ++
 src/audio_output/common.c                    | 209 ++++++++++++++-------------
 src/audio_output/filters.c                   |   4 +-
 src/misc/es_format.c                         |   3 +-
 15 files changed, 194 insertions(+), 105 deletions(-)

-- 
2.9.3