[vlc-devel] [PATCH 1/3] decoder: fix forgotten vout error state

[Steve Lhomme]

On 2020-12-16 18:34, Thomas Guillem wrote:
> If the pool allocation fails or if the vout fails to start (via
> input_resource_RequestVout()), an error is returned from
> decoder_UpdateVideoFormat(). The next call to
> decoder_UpdateVideoFormat() will return a success because the vctx will
> be the same since only this variable is checked to detect a vout change.

You end up with a p_owner->fmt missing the matching vctx.

IMO it should be set/reset in DecoderUpdateFormatLocked() so they are 
always paired.

> To fix this issue, also release the vctx when returning an error.
> ---
>   src/input/decoder.c | 8 +++++++-
>   1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/src/input/decoder.c b/src/input/decoder.c
> index 94e328f040e..b9cdc7d4a81 100644
> --- a/src/input/decoder.c
> +++ b/src/input/decoder.c
> @@ -457,7 +457,7 @@ static int ModuleThread_UpdateVideoFormat( decoder_t *p_dec, vlc_video_context *
>               msg_Err(p_dec, "Failed to create a pool of %d %4.4s pictures",
>                              dpb_size + p_dec->i_extra_picture_buffers + 1,
>                              (char*)&p_dec->fmt_out.video.i_chroma);
> -            return -1;
> +            goto error;
>           }
>   
>           vlc_mutex_lock( &p_owner->lock );
> @@ -481,6 +481,12 @@ static int ModuleThread_UpdateVideoFormat( decoder_t *p_dec, vlc_video_context *
>           return 0;
>       }
>   
> +error:
> +    if (p_owner->vctx != NULL)
> +    {
> +        vlc_video_context_Release(p_owner->vctx);
> +        p_owner->vctx = NULL;
> +    }
>       return -1;
>   }
>   
> -- 
> 2.29.2
> 
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
>