[vlc-devel] [RFC PATCH 0/9] Fixing A/V dejitter with high latency devices

[Thomas Guillem]

Hello,

Playing a video/audio media when using a Bluetooth device can lead to
1-20 frame drops at the beginning. It happens with PulseAudio and
possibly on all other aout modules (and possibly all kind of audio
devices).

I explain. The video output and the audio have a minimum dejiiter time
of 80 ms (it can be higher with some inputs). It means that both the
video and the audio will wait 80ms before starting. Ideally, the audio
latency should be known before this dejitter delay so that the vout is
rescheduled via the clock before displaying its first frame.

It seems it is never the case with most aout modules. Indeed, the audio
delay is generally known after 160ms (or more). Therefore, the vout,
that started to display some frames, has to drop a few frames to catch
up with the new delay. It means a video glitches for us.

Actually, the audio delay could be known before 80ms (but it's rarely
the case), it totally depends on the input/buffering/decoder speed.
Indeed, a burst of data is first sent to the aout via the audio decoder
thread, then this thread is put to sleep waiting for more data (or he is
decoding new data), causing the next time_get() to be called way too
late, once a new data is available.

The more robust way to fix this issue is to propagate the audio delay
from all aout modules asynchronously, not depending on how the input is
buffered/decoded.

This patch set implement this solution for Pulse Audio. This module
didn't require many modifications to achieve that purpose since the
PulseAudio provide a callback to be notified when the latency is
updated.

Other modules will be more problematic to fix. I see 2 solutions:

1/ Writing a threaded time_get helper code. An aout module could use
this helper to create a thread that will poll the delay via a time_get
callback. During the first phrase, this delay could be fetched every
20ms, and then fetched every 1 seconds once the stream is stabilized.

2/ Add a callback to the master clock that will fetch the aout delay.
This callback could be triggered by the vout before displaying a frame.

Both solutions will need some work on protecting shared data.

Questions:
 - Are you OK with the actual PulseAudio fixes?
 - Which solution do you prefer for other modules? 1 or 2?

Best regards,
Thomas Guillem (9):
  aout: reorder
  aout: split aout_RequestRetiming()
  aout: fix data races with aout_RequestRetiming()
  aout: add aout_TimingReport helper
  aout: time_get is not mandatory anymore
  pulse: log buffer_attr in us
  pulse: Drain: use vlc_pa_get_latency() directly
  pulse: request timing asynchronously
  pulse: don't interpolate timing

 include/vlc_aout.h               |   9 +-
 modules/audio_output/pulse.c     |  54 +++++------
 src/audio_output/aout_internal.h |   3 +-
 src/audio_output/dec.c           | 162 ++++++++++++++++++-------------
 src/audio_output/output.c        |   5 +-
 5 files changed, 132 insertions(+), 101 deletions(-)

-- 
2.29.2