[vlc-devel] [PATCH] [RFC] picture: integrate the 2 extra lines in the height padding

[Steve Lhomme]

This is a rebased/reworked version of 
https://patches.videolan.org/patch/19128/

Also since this is done in picture_Setup() I wonder if we should also 
update the i_width/i_height of the picture accordingly. Right now a 
picture that has its height modified for padding has its plane lines 
correct but the height reported doesn't match. Leading to potential 
misdetection of padding/alignment.


Le 05/04/2018 à 08:51, Steve Lhomme a écrit :
> No need for extra lines when there's already padding. Plus it may break the
> alignment of the secondary planes.
> ---
>   src/misc/picture.c | 10 +++++-----
>   1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/src/misc/picture.c b/src/misc/picture.c
> index 7506e47725..7ce1170702 100644
> --- a/src/misc/picture.c
> +++ b/src/misc/picture.c
> @@ -133,17 +133,17 @@ int picture_Setup( picture_t *p_picture, const video_format_t *restrict fmt )
>   
>       unsigned width, height;
>   
> +    /* Hack: append two scan lines for some SIMD assembler */
> +    if (unlikely(add_overflow(fmt->i_height, 2 * i_ratio_h, &height)))
> +        return VLC_EGENERIC;
> +
>       if (unlikely(add_overflow(fmt->i_width, i_modulo_w - 1, &width))
> -     || unlikely(add_overflow(fmt->i_height, i_modulo_h - 1, &height)))
> +     || unlikely(add_overflow(height, i_modulo_h - 1, &height)))
>           return VLC_EGENERIC;
>   
>       width = width / i_modulo_w * i_modulo_w;
>       height = height / i_modulo_h * i_modulo_h;
>   
> -    /* Hack: append two scan lines for some SIMD assembler */
> -    if (unlikely(add_overflow(height, 2 * i_ratio_h, &height)))
> -        return VLC_EGENERIC;
> -
>       /* plane_t uses 'int'. */
>       if (unlikely(width > INT_MAX) || unlikely(height > INT_MAX))
>           return VLC_EGENERIC;