[vlc-devel] [RFC PATCH] subpicture: YUVP subpicture formats should have a valid palette

[Steve Lhomme]

On Tue, May 3, 2016 at 11:58 AM, Thomas Guillem <thomas at gllm.fr> wrote:
> And return NULL if the palette allocation fails since most callers will assume
> that palette will be valid.
>
> Question:
> Maybe the palette allocation should be done in video_format_Setup when color is
> YUVP, no ?

I'd say even in video_format_Init(). But none of them return an error.
video_format_Init() is often called with a 0 chroma (I wonder what's
the parameter for, then).

> ---
>  src/misc/subpicture.c | 20 +++++++++++++++++---
>  1 file changed, 17 insertions(+), 3 deletions(-)
>
> diff --git a/src/misc/subpicture.c b/src/misc/subpicture.c
> index 7f5d109..3ea95d8 100644
> --- a/src/misc/subpicture.c
> +++ b/src/misc/subpicture.c
> @@ -206,12 +206,26 @@ subpicture_region_t *subpicture_region_New( const video_format_t *p_fmt )
>      if( !p_region )
>          return NULL;
>
> -    video_format_Copy( &p_region->fmt, p_fmt );
> -    if ( p_fmt->i_chroma != VLC_CODEC_YUVP )
> +    if ( p_fmt->i_chroma == VLC_CODEC_YUVP )
>      {
> -        free( p_region->fmt.p_palette );
> +        video_format_Copy( &p_region->fmt, p_fmt );
> +        /* YUVP should have a palette */
> +        if( p_region->fmt.p_palette == NULL )
> +        {
> +            p_region->fmt.p_palette = calloc( 1, sizeof(*p_region->fmt.p_palette) );
> +            if( p_region->fmt.p_palette == NULL )
> +            {
> +                free( p_region );
> +                return NULL;
> +            }
> +        }
> +    }
> +    else
> +    {
> +        p_region->fmt = *p_fmt;
>          p_region->fmt.p_palette = NULL;
>      }
> +
>      p_region->i_alpha = 0xff;
>
>      if( p_fmt->i_chroma == VLC_CODEC_TEXT )
> --
> 2.8.0.rc3
>
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel