[vlc-devel] Correcting misinformation (Was: VLC ignores PTS in PES streams)

[David Flynn]

This thread has had a couple of things that will confuse people in the
future.

Firstly, there is confusion over terms.
 - PES is a Packetized Elementry Stream, this contains presentation and
   decode time information

Multiple PES streams are multiplexed into either
 - PS (Programme Stream)
 - TS (Transport Stream)
Both of these have methods of providing a system clock, which is used to
synchronize presentation of multiple PES.

If you want to do things like play stuff backwards, you really should
investigate trickmode.

On 2009-01-17, Matthias Krause <odikrause at gmail.com> wrote:
> PTS is based on the presentation time, DTS on the decoding time.

Not 'based on', but 'is':

PTS is the Presentation TimeStamp, as related to the System Time Clock.
DTS is the Decode TimeStamp, as related to the same clock.

> the DTS should be a little earlier than the PTS (to give the decoder some time)

No, this is not the case.  The timestamps are modeled based upon an
idealized decoder as specified in the standard; the ideal decoder takes
no time to transfer data between buffers or decode.  Ie:
 - There should not be a constant component to the offset (pts - dts).
 - If you wish to increase the buffering, you add a constant to the
   the system time clock.
 - Picture decoding time is constant for all pictures.  therefore, it is
   upto each decoder to take that into account internally.

> The PTS is based on the presentation order of the MPEG stream which
> differs from the transport order

Correct; PTS increments by one picture period for each frame in
presentation order.  Frames at the input to a decoder may not be
in presentation order (But, they may be).

>                                  with a regular pattern of 3, except

No, there are no constants in this.  There can be more or less B frames
(even none at all), which will alter the pattern.  However, modern
encoders have the ability to restart a group of pictures if it thinks
that would be benefitial, so the pattern can have discontinuities.

> for the very first I-frame, where PTS=DTS

No, for the GOP structure you reference, this would never be the case.
It would only be true for either I-frame-only or IP-frame or P-frame-only
coding, ie where there isn't any reordering.

For all types of sequence, PTS === DTS only on non-reordered frames.
For a sequence with reordering, the non-reordered frames are the B-frames.

An example will help:

decoder
 Input DTS: -1,  0,  1,  2,  3,  4,  5,
 Input PTS: I0, P3, B1, B2, P6, B4, B5,
Output PTS:     I0, B1, B2, P3, B4, B5, P6

Let's deconstruct this.  Our decoder has just decoded I0, If it were to
output it immediately, I0 would have to be displayed for two picture
periods. (The period of I0, and the period while P3 is decoded).  To
avoid this, the decoder doesn't ouput anything for the first picture
period.  But this means that the presentation time and the decode time
for I0 aren't the same; infact they differ by 1.

It becomes slightly more obvious as to why when you consider this as a
steady state problem, rather than a start-up problem.

If you're unhappy with the above, given the sequence of input PTS, try
and number the sequence in any way which produces a valid set of DTS.

> I hope thats a bit clear.

Please say if this isn't.