[vlc-devel] [PATCH] vout: fix a deadlock with the picture pool lock

[Felix Abecassis]

ping, I'm afraid this separate patch went unnoticed in the flow of
stereo 3D patches :)

I admit this patch might not be ideal, removing the
ThreadDisplayPicture loop and modifying how the vout thread
sleeps/wakes might be a better solution.

2014-09-09 19:50 GMT+02:00 Felix Abecassis <felix.abecassis at gmail.com>:
> A deadlock could occur in an high load situation where the vout and
> the decoder are taking excessive amounts of time.
>
> The vout thread repeatedly call ThreadDisplayPicture in its main loop
> until it returns an error, while keeping the picture pool locked. If
> no picture was recently received, the vout will redisplay the current
> picture (a "refresh") by calling ThreadDisplayRenderPicture with
> is_forced=true. If this refresh is excessively long, the vout thread
> will be stuck in a refresh loop. The decoder cannot make any progress
> since the picture pool lock is hold and the vout won't be polling for
> control commands, yielding a total deadlock of the program.
>
> This situation can be reproduced artificially by sleeping in the
> decoder and decreasing variable VOUT_REDISPLAY_DELAY.
>
> A simple solution to this issue is to exit the ThreadDisplayPicture
> loop after refreshing. Since a refresh typically occurs when no new
> pictures are received from the decoder, this should not decrease
> performance.
> ---
>  src/video_output/video_output.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/src/video_output/video_output.c b/src/video_output/video_output.c
> index 3f094d4..3577811 100644
> --- a/src/video_output/video_output.c
> +++ b/src/video_output/video_output.c
> @@ -1112,6 +1112,7 @@ static int ThreadDisplayPicture(vout_thread_t *vout, mtime_t *deadline)
>          date_refresh = vout->p->displayed.date + VOUT_REDISPLAY_DELAY - render_delay;
>          refresh = date_refresh <= date;
>      }
> +    bool force_refresh = !drop_next_frame && refresh;
>
>      if (!first && !refresh && !drop_next_frame) {
>          if (!frame_by_frame) {
> @@ -1133,8 +1134,9 @@ static int ThreadDisplayPicture(vout_thread_t *vout, mtime_t *deadline)
>          return VLC_EGENERIC;
>
>      /* display the picture immediately */
> -    bool is_forced = frame_by_frame || (!drop_next_frame && refresh) || vout->p->displayed.current->b_force;
> -    return ThreadDisplayRenderPicture(vout, is_forced);
> +    bool is_forced = frame_by_frame || force_refresh || vout->p->displayed.current->b_force;
> +    int ret = ThreadDisplayRenderPicture(vout, is_forced);
> +    return force_refresh ? VLC_EGENERIC : ret;
>  }
>
>  static void ThreadDisplaySubpicture(vout_thread_t *vout,
> --
> 1.9.1
>



-- 
Félix Abecassis
http://felix.abecassis.me