[vlc-devel] LIVE (low latency) decoding with various VLC version.

[David Glaude]

Dear developper,

We tested various version of VLC for a "special project" where we need low
latency decoding.
Here are our finding based on behaviour and log message observation.
I don't have much inside knowledge of VLC so I might have miss a few obvious
option.

If anybody know the obvious parameter to use in 1.1.0-git to fix our problem
or where to dig... please speak.

Thanks in advance.

David Glaude

===

Goal:

Find a version in wich IE plugin is
* Low latency = there is a minimum delay between receiving the data and
displaying it
* Stable = no memory leak and no crash
* Robust = there is not latency variation over time if some packets (from 1
to x minutes) are lost
* long lasting = can run for days without problem and if required support
many IE page refresh

We tested the following version of VLC:
(A) VLC-0.8.6i-win32.exe (default parameter)
(B) VLC-1.0.5-win32.exe (default parameter)
(C) VLC-1.1.0-git-20100305-0002-win32.exe (default parameter)
(D) VLC-1.1.0-git-20100305-0002-win32.exe (with "--clock-jitter=0")
(E) VLC-1.0.2-win32.exe (default parameter)

Crash test:

(1) Start VLC in plugin and in player mode
(2) Display message and debug level 2
(3) Check the latency (compare the decoded picture to the source)
(4) Unplug the network cable of the encoder (or VLC) for short and long time
(5) Observe the latency variation and behaviour = time to stop, time to
restart, latency at restart, log message
(6) Press F5 many time and observe = does it restart always? does it eat
memory? does it keep the latency?
(7) Let VLC run for hours and days
(8) Observe memory usage of IE and VLC over time (mostly peak value)

Our stream:

QCIF Video only (no sound) MP4 (part 2) video inside UDP multicast send with
RTP and describe by an SDP file either local or HTTP downloaded

Here are the result:

VLC-1.0.2-win32.exe (E) give us the best result for the low latency and for
maintaining that low latency over time and despite packet loss.
Some change occured between 1.0.2 and 1.0.5 that change the behaviour and
introduce additionnal latency.
The problem is not fully solved with "--clock-jitter=0" in 1.1.0-git and
there must be other buffer involved that were not triggered before.
Presentation buffer? Decoding buffer? RTP Buffer?
Something is else is dynamic and change the buffering when we cut the
stream.

Our possible options:
* Find the trick or solution within 1.1.0-git
* use 1.1.0 and force frequent refresh (30 minutes/1 hour) to resync if
needed
* use VLC 1.0.2

Here are the detail:

(A) With 0.8.6i we encountered a crash (picture freeze) after a few hours of
continuous streaming.
Since that version is old (but stable) the chance to have a long term
support are small so that path was abandonned.

(B) With 1.0.5 we found a lot of stability and low latency, but after a few
days we noticed that the latency did increase over time.
To try the robustness of VLC we created artificial loss of connectivity
between the encoder (network cable unplug).
We found that between VLC 1.0.2 and 1.0.3 an feature was added in VLC to
dynamicaly compute a value for "pts_delay".

main error: ES_OUT_SET_(GROUP_)PCR  is called too late, increasing pts_delay
to 300 ms
main error: ES_OUT_RESET_PCR called
main error: ES_OUT_SET_(GROUP_)PCR  is called too late, increasing pts_delay
to 9023 ms
main error: ES_OUT_RESET_PCR called
main error: ES_OUT_SET_(GROUP_)PCR  is called too late, increasing pts_delay
to 30000 ms
main error: ES_OUT_RESET_PCR called

(C) At our request a VLC developper added into the source code tree of VLC
1.1.0 a parameter to disable that new feature.
We were able to test VLC-1.1.0-git-20100305 with or without the new option
"--clock-jitter=0".
Without "--clock-jitter=0" the behaviour is similar to previously

(D) By using the same version with the option "--clock-jitter=0"
(simultaniously) we did notice the change but other source of latency are
present.
It return to low latency behaviour when we see those message in the log:
main warning: clock gap, unexpected stream discontinuity
main warning: feeding synchro with a new reference point trying to recover
from clock gap

However there are time where there is a sometime huge buffering occuring
somewhere else (pts_delay) despite the "--clock-jitter=0":
main debug: Stream buffering done (3039 ms in 2348 ms)
main debug: Stream buffering done (4359 ms in 837 ms)

The main difference between --clock-jitter=0 or not is here:

With:
main error: ES_OUT_SET_(GROUP_)PCR  is called too late (jitter of 6016 ms
ingnored)
main error: ES_OUT_RESET_PCR called
main error: ES_OUT_SET_(GROUP_)PCR  is called too late (jitter of 65270 ms
ingnored)
main error: ES_OUT_RESET_PCR called
main error: ES_OUT_SET_(GROUP_)PCR  is called too late (jitter of 3008 ms
ingnored)
main error: ES_OUT_RESET_PCR called

Without:
main error: ES_OUT_SET_(GROUP_)PCR  is called too late (pts_delay increased
to 1804 ms)
main error: ES_OUT_RESET_PCR called
main error: ES_OUT_SET_(GROUP_)PCR  is called too late (jitter of 65270 ms
ingnored)
main error: ES_OUT_RESET_PCR called
main error: ES_OUT_SET_(GROUP_)PCR  is called too late (pts_delay increased
to 2023 ms)
main error: ES_OUT_RESET_PCR called

(E) Currently the best result (at least for robustness) is
VLC-1.0.2-win32.exe. We don't know yet the result after 2 or 3 days.
But whenever the video restart after the network connectivity is restored,
it is always with low latency.

On occasion we see the following message (not always) when we force a
connectivity loss:
main warning: clock gap, unexpected stream discontinuity
main warning: feeding synchro with a new reference point trying to recover
from clock gap

There is no additionnal message in the log except for some message always
present under normal condition and low input buffer.
main warning: late picture skipped (8842 > -956)
main warning: late picture skipped (128124 > -334)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20100308/4959400a/attachment.html>