[vlc-devel] [PATCH v2 2/4] video_output: rework the decision to use the next picture to display

[Steve Lhomme]

Until now, in normal playback, the way to skip to the next frame to display was
* get displayed.current
* get displayed.next
* if displayed.next has time to render use it to replace displayed.current
* the vout thread waits until the deadline to render displayed.current

Now we don't use displayed.next anymore. We keep the displayed.current has long
as the "time to render" (the deadline to render - current time) is bigger than
the time it takes to render (approximately render_delay).
If there is not enough time, we try to get the next picture from the
decoder+prerender. And we check if there's enough time to render that one.
If there is not enough time, we try to get the next picture from the
decoder+prerender. And so on...

This allows having one picture about to be displayed in memory rather than 2
(displayed.current and displayed.next) which should be useful to decide when to
change the display module (when displayed.current changes, not displayed.next).

If the user disabled the late frame dropping we drop any at this stage, we just
skip to the next one by one.

We also count the number of extra pictures dropped in case many "next" pictures
had to be released before there is one with enough time to be rendered. No need
to try to render a picture that we know will be late if there's a non late one
available. The vout thread is less likely to drop the pictures we return as the
amount of time to filter+render is already taken in account.
---
 src/video_output/video_output.c | 57 +++++++++++++++++++++------------
 1 file changed, 37 insertions(+), 20 deletions(-)

diff --git a/src/video_output/video_output.c b/src/video_output/video_output.c
index cea13d44c4a..c86b4ff865c 100644
--- a/src/video_output/video_output.c
+++ b/src/video_output/video_output.c
@@ -1502,15 +1502,6 @@ static int ThreadDisplayPicture(vout_thread_sys_t *vout, vlc_tick_t *deadline)
     }
     else
     {
-        if (!paused)
-        {
-            if (!sys->displayed.next)
-            {
-                sys->displayed.next =
-                    ThreadDisplayPreparePicture(vout, false, false, &paused);
-            }
-        }
-
         const vlc_tick_t system_now = vlc_tick_now();
         const vlc_tick_t render_delay = vout_chrono_GetHigh(&sys->render) + VOUT_MWAIT_TOLERANCE;
 
@@ -1534,26 +1525,52 @@ static int ThreadDisplayPicture(vout_thread_sys_t *vout, vlc_tick_t *deadline)
         }
         render_now = refresh;
 
-        if (!paused && sys->displayed.next) {
-            const vlc_tick_t next_system_pts =
+        if (!paused) {
+            vlc_tick_t pic_system_pts =
                 vlc_clock_ConvertToSystem(sys->clock, system_now,
-                                        sys->displayed.next->date, sys->rate);
-            if (likely(next_system_pts != INT64_MAX))
+                                          sys->displayed.current->date, sys->rate);
+            if (likely(pic_system_pts != INT64_MAX))
             {
-                vlc_tick_t date_next = next_system_pts - render_delay;
-                if (date_refresh == VLC_TICK_INVALID || date_next < date_refresh)
-                    date_refresh = date_next;
-
-                if (date_next <= system_now)
+                // check if current is late and we need to use the next one(s)
+                vlc_tick_t time_to_render = pic_system_pts - system_now;
+                unsigned skipped_to_next = 0;
+                while (time_to_render <= render_delay)
                 {
+                    // not enough (predicted) time to render current, try to get next pic
+                    picture_t *next;
+                    next = ThreadDisplayPreparePicture(vout, false, false, &paused);
+                    if (next == NULL)
+                        // no next picture, keep using current
+                        break;
+                    skipped_to_next++;
+
                     // next frame will still need some waiting before display
                     dropped_current_frame = true;
                     render_now = false;
 
                     if (likely(sys->displayed.current != NULL))
                         picture_Release(sys->displayed.current);
-                    sys->displayed.current = sys->displayed.next;
-                    sys->displayed.next    = NULL;
+                    sys->displayed.current = next;
+
+                    if (!sys->is_late_dropped)
+                        break;
+
+                    pic_system_pts =
+                        vlc_clock_ConvertToSystem(sys->clock, system_now,
+                                                  sys->displayed.current->date, sys->rate);
+                    if (likely(pic_system_pts == INT64_MAX))
+                        break; // got paused, stick to the current one
+                    time_to_render = pic_system_pts - system_now;
+                }
+
+                vlc_tick_t start_render_deadline = pic_system_pts - render_delay;
+                if (date_refresh == VLC_TICK_INVALID || start_render_deadline < date_refresh)
+                    date_refresh = start_render_deadline;
+
+                if (skipped_to_next > 1)
+                {
+                    msg_Warn(&vout->obj.obj, "dropped %u pictures too late to be displayed", skipped_to_next-1);
+                    vout_statistic_AddLost(&sys->statistic, skipped_to_next-1);
                 }
             }
         }
-- 
2.26.2