[vlc-devel] [RFC-PATCH 00/12] They see me rolling^Wprerolling

[Filip Roséen]

Since this cover-letter is really long, here's an ASCII-ball (since this
patch-batch covers (pre)rolling).


           _...----.._
        ,:':::::.     `>.
      ,' |:::::;'     |:::.
     /    `'::'       :::::\
    /         _____     `::;\
   :         /:::::\      `  :
   | ,.     /::VLC::\        |
   |;:::.   `::::::;'        |
   ::::::     `::;'      ,.  ;
    \:::'              ,::::/
     \                 \:::/
      `.     ,:.        :;'
        `-.::::::..  _.''
           ```----'''

Introduction
------------

All of these patches are related to how the core handles prerolling, both after
usage of ES_OUT_SET_NEXT_DISPLAY_TIME, but also what happens with blocks
flagged with BLOCK_FLAG_PREROLL.

This entire patch-batch fixes (at least) the following issues:

  - https://trac.videolan.org/vlc/ticket/10422 *
  - https://trac.videolan.org/vlc/ticket/13575
  - https://trac.videolan.org/vlc/ticket/7337
  - https://trac.videolan.org/vlc/ticket/10638
  - https://trac.videolan.org/vlc/ticket/16361

* marked as "fixed", but the video audio used to stutter on my system.

These patches makes sure that:

  - the decoder knows the exact timestamp that belongs to a preroll.
  - the preroll is only based on the PTS of incoming blocks.
  - functionality regarding preroll happens where they should.
  - demuxers can override the core regarding preroll decisions, if
    that is required.

Elaboration
-----------

Given the relationship between ES_OUT_SET_NEXT_DISPLAY_TIME, it is not
suprising that the end-of preroll timestamp have a tight relationship
with the PTS of blocks that pass through the core.

Inside the decoders the core would handle the end-of preroll timestamp
as if it simply could be in two states:

  - Either we are prerolling, or;
  - we are not prerolling.

Sure, the es_out should be responsible for saying whether we are
prerolling or not, but there are several cases where the decoder must
know exactly where we are prerolling to (because decisions whether
something is part of the preroll or not cannot be made until after
decoding has happend).

The core previously made the decision whether it should mark an entity
as part of the preroll prematurely. Incoming blocks does not necessary
have a PTS, and we can only decide what to do with these blocks after
the entity has been decoded (since the underlying decoder should know
the real PTS).


  Matroska
  --------

    When working on the matroska demuxer I noticed many issues related to
    broken audio after seeking because of the core wrongfully marking
    blocks that should be played as being part of the preroll.

    The matroska file format does not allow for the demuxer to get the PTS
    of every audio block, but instead only the PTS of an entire "audio
    block group".

    This means that the demuxer will send the core a bunch of blocks where
    block_t.i_pts has been set to VLC_TS_INVALID (since the demuxer cannot
    know the exact PTS for the blocks within the block-group).

    [ NOTE: not sending a value for i_pts/i_dts is ok, and documented
    in include/vlc_block.h ]

    When these blocks hit the core during a preroll, the core would
    checked whether the block_t.i_pts was less than the end-of-preroll
    timestamp; since this was true, the blocks would be discarded
    (causing broken audio after seek).

    By post-poning the actual decision for blocks that have a missing
    PTS until after the decoder has been invoked the problem is
    solved.

  Subtitles
  ---------

    When seeking, subtitles that span across the end-of preroll
    timestamp should be played (in order to display the subtitle of
    where you seeked), but previously such subtitles would be dropped
    because of errors in how the preroll (and subtitles) was handled.

Filip Roséen (12):
  core: changed the type of decoder_owner_sys_t.i_preroll_end
  core: removal of DecoderUpdatePreroll
  core: mark blocks as part of preroll if they have a valid PTS
  core: introduce input_Decoder{Set,Get}PrerollEnd
  core: added EsOutDecodersSetPreroll to es_out.c
  core: added BLOCK_FLAG_DISPLAY_FORCED
  core: introduced DecoderPrerollBlock
  core: refactored prerolling related to video
  core: refactored prerolling related to audio
  core: added DecoderUpdateStatSpu
  core: simplified DecoderQueueSpu
  core: refactored DecoderProcessSpu

 include/vlc_block.h |   2 +
 src/input/decoder.c | 251 +++++++++++++++++++++++++++++-----------------------
 src/input/decoder.h |  17 ++++
 src/input/es_out.c  |  40 ++++++---
 4 files changed, 188 insertions(+), 122 deletions(-)

-- 
2.8.2