[vlc-devel] VLC Support for RTCP extensions anb playback buffer management

[Mario Montagud Climent]

Dear VLC Developers,

This is my first message to this mailing list.

My name is Mario Montagud. I am a PhD student at Politechnic  
University of Valencia (UPV) in Spain. I am working under Prof.  
Fernando Boronat supervision.  
(http://personales.gan.upv.es/~fboronat/Research/index_investig_en.html).

I would like to ask you some information about the ?netsync? module in  
VLC and the VLC capabilities.

My PhD work is about Multimedia Communications and Synchronization  
Techniques. Concretely, our research group is designing an  
Inter-Destination Multimedia Synchronization (IDMS) proposal that aims  
to synchronize the playout of media streams across separated clients  
(which can be placed in different networks, with different network  
latency). For that purpose, we take advantage of the feedback and  
extension capabilities of RTP/RTCP protocols.

You can find more information about the preliminary version of our  
IDMS approach in the reference [Bor08], in which it was implemented by  
using VIC (for video) and RAT (for audio) mbone tools, and tested in a  
real scenario (our University WAN).

In the initial stages of my PhD, I have implemented this IDMS approach  
in a network simulator (NS-2), and I?ve added some enhancements to  
that solution (see [Bor11]). The simulation results have been  
satisfactory.

Currently, an evolved version of our IDMS proposal is under  
standardization status within the IETF (see ref [Bra11]). We are  
collaborating in its specification/design in conjunction with some  
Dutch researchers from TNO (Delft).

On the one hand, a new RTCP report block (RTCP XR for IDMS block) has  
been specified to inform the source (or a sync manager) about receipt  
and playout times for specific RTP media packets (for that purpose all  
the participants must share a common available clock source, e.g.  
NTP). On the other hand a new RTCP packet type (RTCP IDMS packet) has  
been defined to send playout setting instructions to the sync clients  
(if an out of sync situation is detected by the sync manager).

My next objective is to implement and test that IDMS proposal in a  
real media framework/player. So, I?m searching for the most  
appropriated solution for it.

I am a newbie in VLC media player (as a developer). For that reason, I  
send this mail asking for your recommendation.

I know that the ?netsync? module in VLC can be used to keep several  
clients synchronised on a single VLC stream (using UDP datagrams, not  
using RTP/RTCP). It can also be used to synchronise several computers  
playing parts of a video wall.

Our goal is to synchronize a RTP network stream across multiple  
separated clients. We use for that purpose the global time instant  
(64-bit wall clock time, eg. NTP) at which a specific RTP packet has  
been received (or played out) by each sync client.

For that purpose, first, I need to implement the new RTCP packet types  
and block reports defined in [Bra11]. I think it should be done in  
live555 library.

I think the most difficult task will be the management of the playout  
processes in each media client to get in sync (e.g. to report on  
presentation time for a specific RTP packet ?e.g. the last one  
encapsulating a video frame-, and to manage the necessary playback  
adjustments (playout speed variation? Skips/pauses?).

Do you think VLC is a good choice for implementing our RTP/RTCP-based  
IDMS solution? Can I access to the playback buffer to know about  
presentation times of a specific video frame and the current playback  
buffer occupancy?

Could the "netsync" module be adapted for our IDMS purposes?

Your comments will be much appreciated.

Thank you very much in advance.

Regards from Spain,

Mario

PD: I apologize for the extension of the mail.

REFERENCES: (If you are interested, and don?t have access to them,  
please, tell me)

[Bor08] F. Boronat, J.C. Guerri, and J. Lloret, ?An RTP/RTCP based  
approach for multimedia group and inter-stream synchronization?,  
Multimedia Tools and Applications Journal, Vol. 40 (2), 285-319, June  
2008.

[Bor11] F. Boronat, M. Montagud, and V. Vidal, ?Smooth Control of  
Adaptive Media Playout to Acquire IDMS in Cluster-based Applications?,  
IEEE LCN 2011, Bonn, October 2011.

[Bra11] R. V. Brandenburg, H. Stokking, M.O. V. Deventer, Boronat, F.  
M. Montagud, K. Gross, ?RTCP for Inter-Destination Media  
Synchronization?, IETF Internet Draft, October 2011.