[vlc-devel] Problems with synchronizing video and audio using RTP (live555)

[Jami Pekkanen]

Hello,

I have a setup with a homebrew RTP client sending audio (AMR) and video
(MP4-ES) stream and a VLC (current trunk and live555 from yesterday)
instance playing it using a SDP file to initialize the playing (ie $ vlc
stream.sdp). When playing the streams separately, it works fine, but
when playing them at the same time, the synchronization fails (constant
"PTS is out of range" for audio). This leads to video playing fine, but
with audio mostly missing. It does work if the playing starts from the
start of the stream, but if too much frames are dropped, this falls out
of sync also.

I've also tried to synchronize the streams using RTCP. In this case
before any RTCP have arrived, the behavior is same as without RTCP at
all (quite expected), but when the first RTCP packet arrives, the audio
gets messed up. Sometimes it stays at "buffering 0%" and sometimes
complains about "buffer in the future". Also in this case playing video
only works fine with RTCP synchronization. However, audio, which works
fine alone without RTCP, gets messed up similarly than the full stream
when it gets the first RTCP SR. I've noticed that for video stream
resets  the PCR with ES_OUT_RESET_PCR, but the audio stream fails to do
this. Also the message "tk->rtpSource->hasBeenSynchronizedUsingRTCP()"
is missing for the audio stream.

The stream data itself should be quite fine as it works at least with
Quicktime.

I've tried to debug this (mostly in live555.c module and live555 code),
but I'm having some problems with GDB: stepping in the code seems to
jump all over the function in eg live555.c:StreamRead. And it's a bit
too complex to figure out without a debugger.

Are there some oddities in the AMR synchronization? Also pointers where
to start digging in to this (and how to do the debugging) are welcome.

- Jami Pekkanen