[vlc-devel] [PATCH 3/5] vout: chroma chain: fix scaling before transform

[Steve Lhomme]

On 2019-09-10 16:11, Francois Cartegnie wrote:
> ---
>   modules/video_chroma/chain.c | 7 +++++--
>   1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/modules/video_chroma/chain.c b/modules/video_chroma/chain.c
> index 6cb9589721..8b183d2686 100644
> --- a/modules/video_chroma/chain.c
> +++ b/modules/video_chroma/chain.c
> @@ -295,8 +295,11 @@ static int BuildTransformChain( filter_t *p_filter )
>           return VLC_SUCCESS;
>   
>       /* Lets try resize+chroma first, then transform */
> -    msg_Dbg( p_filter, "Trying to build chroma+resize" );
> -    EsFormatMergeSize( &fmt_mid, &p_filter->fmt_out, &p_filter->fmt_in );
> +    msg_Dbg( p_filter, "Trying to build chroma+resize, then transform" );
> +    es_format_Copy( &fmt_mid, &p_filter->fmt_out );
> +    /* Set mid format to same rotation as source,
> +     * so we get proper scaling before transform */
> +    video_format_TransformTo( &fmt_mid.video, p_filter->fmt_in.video.orientation );

This is not equivalent to what EsFormatMergeSize() was doing. For 
example the fmt_in SAR was copied to the middle format. Same thing for 
the fmt_in dimensions which are not used at all here.

Expanding EsFormatMergeSize() is probably the way to go.

>       i_ret = CreateChain( p_filter, &fmt_mid );
>       es_format_Clean( &fmt_mid );
>       if( i_ret == VLC_SUCCESS )
> -- 
> 2.21.0
> 
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
>