[vlc-devel] [PATCH] Option to ignore PCR jitter
David R Robison
drrobison at openroadsconsulting.com
Wed Sep 18 15:34:06 CEST 2013
Our problem is that we are trying to play MPEG video with minimal
buffering. This is because our operators are trying to control the video
with a joystick while watching the decoded video on their screen. Any
delay due to buffering can make it difficult for them to accurately
control the cameras. The video is being transported using multicast.
What we see is jitter in when the packets actually arrive to our machine
and into VLC. For example, from the VLC log, the following would
indicate that the PCR arrived 17ms late (as the original buffer was set
to 200ms)
main error: ES_OUT_SET_(GROUP_)PCR is called too late (pts_delay
increased to 217 ms)
main error: ES_OUT_RESET_PCR called
main debug: Buffering 0%
main debug: End of video preroll
main debug: Received first picture
main debug: Buffering 61%
main debug: Buffering 92%
main debug: Stream buffering done (266 ms in 55 ms)
The problem our operator sees is that the video will go black and
rebuild as the new packets arrive. This is because of the resetting of
the PCR and the dumping of the buffer in order to resize it.
We also see where the amount of buffering is not always predictable. It
appears that the stream is buffering and the program checks on the
buffer periodically (maybe with each PCR). This can cause over buffering
as the following from the VLC log shows:
main error: ES_OUT_SET_(GROUP_)PCR is called too late (pts_delay
increased to 381 ms)
main error: ES_OUT_RESET_PCR called
main debug: Buffering 0%
main debug: Buffering 0%
main debug: Buffering 0%
main debug: Buffering 0%
main debug: Stream buffering done (2202 ms in 1025 ms)
We wanted 381ms of buffering but got 2202ms instead.
In our application, since we are watching live video, we don't care as
much about clock synchronization or the timing of each packet as much as
we do about video being decoded (the buffer not being reset) and there
being minimal delay between receiving the video and playing the video.
In fact, I had also thought of scanning the decoder FIFO looking for the
number of IFrames and, if there is more than one, then just skipping to
the last IFrame. However, this approach seemed easier at the moment.
Anyway, the change I proposed basically just accepts what ever jitter
there is on the PCR and will not attempt a reset of the decoder's PCR
nor try to extend the current buffer size.
We have tested this by using a connection with limited bandwidth which
introduced jitter in the video packet delivery. When playing the video
with QuickTime (what the camera's web page uses) we see part of the
picture being discarded and redrawn repeatedly. Also, in VLC we see the
buffer being enlarged with the image going black and refilling with the
larger buffer. However, with this fix, the packets kept decoding
regardless of the jitter in the PCR.
I know this is a lot of info and hopefully it answered your question. If
not, I'd be happy to clarify further.
David
David R Robison
Open Roads Consulting, Inc.
103 Watson Road, Chesapeake, VA 23320
phone: (757) 546-3401
e-mail: drrobison at openroadsconsulting.com
web: http://openroadsconsulting.com
blog: http://therobe.blogspot.com
book: http://www.xulonpress.com/bookstore/bookdetail.php?PB_ISBN=9781597816526
On 9/18/2013 9:13 AM, Rafaël Carré wrote:
> Le 18/09/2013 14:36, David Robison a écrit :
>> Any thoughts on this proposal? David
> Hello David,
>
> Sadly this code is unmaintained and a bit undecipherable.
>
> I remarked that there are a lot of TODO in this function, any chance
> they are related to your problem?
>
> Specifically I wonder if the PCR (Program Clock Reference, not PRC !) in
> your stream is correct? Are you using Transport Stream?
>
> This problem could show a deeper problem in our clock code which has not
> been touched in years so I prefer to get a better understanding of
> what's going on before applying a work around.
>
> Thank you,
>
>> _____
>>
>> From: David Robison [mailto:drrobison at openroadsconsulting.com]
>> To: vlc-devel at videolan.org
>> Sent: Fri, 13 Sep 2013 08:43:29 -0400
>> Subject: [vlc-devel] [PATCH] Option to ignore PRC jitter
>>
>> We are playing live video encoded at the cameras and sent over the network as multicast traffic. We are also trying to run with minimal buffering. This is to make the video seem more "responsive" when the user uses the software to pan/tilt/zoom the camera. With high buffering the user may pan the camera and wait seconds before they actually see it move. We are seeing a lot of jitter in the data feed and the PRC is resetting often which causes 1) the video to momentarily go black and then "rebuild" as the video "trickles" in, and 2) causes the buffering to increase. Normally the buffer increase is small, for example, growing from 200ms to 270ms. However, at times when jitter is a problem, the buffer can grow expectantly large. For example, the following VLC log shows that the buffer was attempting to increase to 381ms but actually got increased to 2202ms. At times we have even seen the buffer grow to over 4 seconds of video.
>>
>> main error: ES_OUT_SET_(GROUP_)PCR is called too late (pts_delay increased to 381 ms)
>> main error: ES_OUT_RESET_PCR called
>> main debug: Buffering 0%
>> main debug: Buffering 0%
>> main debug: Buffering 0%
>> main debug: Buffering 0%
>> main debug: Stream buffering done (2202 ms in 1025 ms)
>> main debug: End of video preroll
>> main debug: Received first picture
>> main debug: Decoder buffering done in 0 ms
>>
>> The proposed patch is to allow the PRC reset and re-buffering to be disabled when the "clock-jitter" option is set to zero. When playing live video we are not as concerned with synchronicity of the video as much as we are the size of the buffer. We made these changes and tested the video for a prolonged period of time and it responded as we expected. We were also able to replicate the issue of high jitter by using a bandwidth limited connection and actually saw the video artifacts in QuickTime while VLC with these changes played the video without incident.
>>
>> Does this change sound reasonable? Could I be missing something by not resetting the PRC and not flushing the buffer? Any other comments?
>> Thanks, David
>>
>>
>> ---
>> diff --git a/src/input/es_out.c b/src/input/es_out.c
>> index 6b0e9ad..f44e04f 100644
>> --- a/src/input/es_out.c
>> +++ b/src/input/es_out.c
>> @@ -2311,19 +2311,20 @@ static int EsOutControlLocked( es_out_t *out, int i_query, va_list args )
>>
>> if( p_pgrm == p_sys->p_pgrm )
>> {
>> + const mtime_t i_jitter_max = INT64_C(1000) * var_InheritInteger( p_sys->p_input, "clock-jitter" );
>> if( p_sys->b_buffering )
>> {
>> /* Check buffering state on master clock update */
>> EsOutDecodersStopBuffering( out, false );
>> }
>> - else if( b_late && ( !p_sys->p_input->p->p_sout ||
>> + else if( b_late && i_jitter_max > 0
>> + && ( !p_sys->p_input->p->p_sout ||
>> !p_sys->p_input->p->b_out_pace_control ) )
>> {
>> const mtime_t i_pts_delay_base = p_sys->i_pts_delay - p_sys->i_pts_jitter;
>> mtime_t i_pts_delay = input_clock_GetJitter( p_pgrm->p_clock );
>>
>> /* Avoid dangerously high value */
>> - const mtime_t i_jitter_max = INT64_C(1000) * var_InheritInteger( p_sys->p_input, "clock-jitter" );
>> if( i_pts_delay > __MIN( i_pts_delay_base + i_jitter_max, INPUT_PTS_DELAY_MAX ) )
>> {
>> msg_Err( p_sys->p_input,
>
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
This email communication (including any attachments) may contain confidential and/or privileged material intended solely for the individual or entity to which it is addressed.
If you are not the intended recipient, please delete this email immediately.
More information about the vlc-devel
mailing list