[vlc-devel] VLC Support for RTCP extensions anb playback buffer management

[Rémi Denis-Courmont]

	Hello,

Le vendredi 16 décembre 2011 20:41:19 Mario Montagud Climent, vous avez écrit 
:
> I know that the ?netsync? module in VLC can be used to keep several
> clients synchronised on a single VLC stream (using UDP datagrams, not
> using RTP/RTCP). It can also be used to synchronise several computers
> playing parts of a video wall.

The netsync module can be used for any combination of media sources, so long 
as all sources have consistent time stamps on all nodes (master and slaves).

It was mainly designed to synchronize playback of identical local files, but 
it can be used with multicast RTP too.

> Our goal is to synchronize a RTP network stream across multiple
> separated clients. We use for that purpose the global time instant
> (64-bit wall clock time, eg. NTP) at which a specific RTP packet has
> been received (or played out) by each sync client.
> 
> For that purpose, first, I need to implement the new RTCP packet types
> and block reports defined in [Bra11]. I think it should be done in
> live555 library.

RTCP SR is currently done in live555, yes.

> I think the most difficult task will be the management of the playout
> processes in each media client to get in sync (e.g. to report on
> presentation time for a specific RTP packet ?e.g. the last one
> encapsulating a video frame-, and to manage the necessary playback
> adjustments (playout speed variation? Skips/pauses?).

On a typical media player, the playback speed is typically determined by the 
audio DAC buffer speed. Alternatively, it could be determined by the vertical 
refresh of the display. There is generally no way to tune the speed of the DAC 
or the display, so this cannot work for synchronized playback and/or for a 
live media source.

Thus VLC always synchronizes based the media source/input, not the 
sinks/outputs. If the media is not intrinsically paced, the local RTC is used 
as fallback (this would not the case with RTP though). Either way, this can 
always drift from the audio output sample rate and the display refresh rate, 
so VLC resamples on-the-fly to keep everything in synchronization.

> Do you think VLC is a good choice for implementing our RTP/RTCP-based
> IDMS solution? Can I access to the playback buffer to know about
> presentation times of a specific video frame and the current playback
> buffer occupancy?

There is an input clock, that gets driven by the input as outlined above, 
and/or by the netsync module. Its job is to correlate the input playback 
timestamps to the local RTC timestamps.

Audio samples are pushed to the hardware as early as possible to minimize 
underruns and maximize energy efficiency. The hardware audio device driver 
will provide a latency estimate for audio playback (latency may be longer than 
buffer time). Then the VLC audio plugin will upsample or downsample if latency 
is higher or lower than expected, so that it keeps up with the input intended 
speed. There is no feedback from the output to the input. It is all done 
within the audio output subsystem.

Similarly, the video output subsystem will drop frames if it notices that the 
computer is unable to render all of them fast enough. 

> Could the "netsync" module be adapted for our IDMS purposes?

I am not very familiar with netsync but there is no real answer to that 
question. Coding will be needed anyway.

-- 
Rémi Denis-Courmont
http://www.remlab.net/
http://fi.linkedin.com/in/remidenis