[vlc-devel] [PATCH 1/2] mp4 demux: ensure the PTS is set (to the DTS)

[Rafaël Carré]

Le Sun, 13 Nov 2011 10:58:16 +0100,
Laurent Aimar <fenrir at elivagar.org> a écrit :

> On Sun, Nov 13, 2011 at 12:27:07AM -0500, Rafaël Carré wrote:
> > Also for videos (in case we didn't find the ctts box)
> > ---
> >  modules/demux/mp4/mp4.c |    4 +---
> >  1 files changed, 1 insertions(+), 3 deletions(-)
> > 
> > diff --git a/modules/demux/mp4/mp4.c b/modules/demux/mp4/mp4.c
> > index 9561d0f..a9914e7 100644
> > --- a/modules/demux/mp4/mp4.c
> > +++ b/modules/demux/mp4/mp4.c
> > @@ -738,10 +738,8 @@ static int Demux( demux_t *p_demux )
> >                  i_delta = MP4_TrackGetPTSDelta( tk );
> >                  if( i_delta != -1 )
> >                      p_block->i_pts = p_block->i_dts + i_delta;
> > -                else if( tk->fmt.i_cat != VIDEO_ES )
> > -                    p_block->i_pts = p_block->i_dts;
> >                  else
> > -                    p_block->i_pts = VLC_TS_INVALID;
> > +                    p_block->i_pts = p_block->i_dts;
> 
> Sadly, this is not right. MP4 files can miss the ctts but still have B frames
> (or out of ourder I/P).

Is there some section of the spec which tells about out of order I/P ?
Or can the encoder just choose which ever DTS it wants?

Does decoding the first I frame, then all P frames starting from the
last one displayed even makes any sense?

e.g. 
Display  order: I0 P9 P8 P7 P6 P5 P4 P3 P2 P1
Decoding order: I0 P1 P2 P3 P4 P5 P6 P7 P8 P9

> With this patch, such file will now have a jerky playback.
> 
> IMHO, The only proper way to fix this issue, is to improve the H264 packetizer
> but it will be a lot of work.

How are we supposed to infer display order?

There is a frame_num field in slice header but it seems to correspond to
decoding order only, because I can see:

- frame num = 0  type = I  PTS = 0  DTS = 0
- frame num = 1  type = D  PTS = 2  DTS = 1
- frame num = 1  type = B  PTS = 1  DTS = 2
- frame num = 2  type = D  PTS = 3  DTS = 3
- frame num = 3  type = D  PTS = 4  DTS = 4

(In this case the PTS comes from the MP4 mux but we should ignore it so
we can reconstruct it properly).

-- 
Rafaël Carré