[vlc-commits] vout: fix a deadlock with the picture pool lock

[Felix Abecassis]

vlc | branch: master | Felix Abecassis <felix.abecassis at gmail.com> | Tue Sep  9 19:29:06 2014 +0200| [22c80ce310c58f6730291b4026af9c7c8b38fb63] | committer: Felix Abecassis

vout: fix a deadlock with the picture pool lock

A deadlock could occur in an high load situation where the vout and
the decoder are taking excessive amounts of time.

The vout thread repeatedly call ThreadDisplayPicture in its main loop
until it returns an error, while keeping the picture pool locked. If
no picture was recently received, the vout will redisplay the current
picture (a "refresh") by calling ThreadDisplayRenderPicture with
is_forced=true. If this refresh is excessively long, the vout thread
will be stuck in a refresh loop. The decoder cannot make any progress
since the picture pool lock is hold and the vout won't be polling for
control commands, yielding a total deadlock of the program.

This situation can be reproduced artificially by sleeping in the
decoder and decreasing variable VOUT_REDISPLAY_DELAY.

A simple solution to this issue is to exit the ThreadDisplayPicture
loop after refreshing. Since a refresh typically occurs when no new
pictures are received from the decoder, this should not decrease
performance.

> http://git.videolan.org/gitweb.cgi/vlc.git/?a=commit;h=22c80ce310c58f6730291b4026af9c7c8b38fb63
---

 src/video_output/video_output.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/src/video_output/video_output.c b/src/video_output/video_output.c
index 3f094d4..3577811 100644
--- a/src/video_output/video_output.c
+++ b/src/video_output/video_output.c
@@ -1112,6 +1112,7 @@ static int ThreadDisplayPicture(vout_thread_t *vout, mtime_t *deadline)
         date_refresh = vout->p->displayed.date + VOUT_REDISPLAY_DELAY - render_delay;
         refresh = date_refresh <= date;
     }
+    bool force_refresh = !drop_next_frame && refresh;
 
     if (!first && !refresh && !drop_next_frame) {
         if (!frame_by_frame) {
@@ -1133,8 +1134,9 @@ static int ThreadDisplayPicture(vout_thread_t *vout, mtime_t *deadline)
         return VLC_EGENERIC;
 
     /* display the picture immediately */
-    bool is_forced = frame_by_frame || (!drop_next_frame && refresh) || vout->p->displayed.current->b_force;
-    return ThreadDisplayRenderPicture(vout, is_forced);
+    bool is_forced = frame_by_frame || force_refresh || vout->p->displayed.current->b_force;
+    int ret = ThreadDisplayRenderPicture(vout, is_forced);
+    return force_refresh ? VLC_EGENERIC : ret;
 }
 
 static void ThreadDisplaySubpicture(vout_thread_t *vout,