[vlc-devel] [RFC] Fix for BuildTransformChain in 3.0.8

[John Cox]

Hi

I think there is an issue with BuildTransformChain in 3.0.8 when there is a
filter that can convert from a h/w format (in my case MMAL_OPAQUE) and can
resize but cannot do a transform.  In my case the existing version of this
code creates a filter that converts out of MMAL to RGB without resizing and
then it fails because the builtin in transform filter cannot do the resize.
This patch asks for an intermediate that is the untransformed output and
thus includes the resize.

diff --git a/modules/video_chroma/chain.c b/modules/video_chroma/chain.c
index 885676f534..4125cc2a00 100644
--- a/modules/video_chroma/chain.c
+++ b/modules/video_chroma/chain.c
@@ -280,8 +280,9 @@ static int BuildTransformChain( filter_t *p_filter )
         return VLC_SUCCESS;

     /* Lets try resize+chroma first, then transform */
-    msg_Dbg( p_filter, "Trying to build chroma+resize" );
-    EsFormatMergeSize( &fmt_mid, &p_filter->fmt_out, &p_filter->fmt_in );
+    msg_Dbg( p_filter, "Trying to build chroma+resize, then transform" );
+    es_format_Copy( &fmt_mid, &p_filter->fmt_out );
+    video_format_TransformTo(&fmt_mid.video, p_filter->fmt_in.video.orientation);
     i_ret = CreateChain( p_filter, &fmt_mid );
     es_format_Clean( &fmt_mid );
     if( i_ret == VLC_SUCCESS )

The code now matches the existing comment and in my case this means I get
a functional filter chain rather than not getting one.

Regards

John Cox