[vlc-commits] swscale: all 4 planes must be set, even if alpha is dropped

[Tristan Matthews]

vlc | branch: master | Tristan Matthews <tmatth at videolan.org> | Thu Sep  3 21:12:11 2015 -0400| [d6e400f2505165a1afd10a1bb6925d9772166077] | committer: Tristan Matthews

swscale: all 4 planes must be set, even if alpha is dropped

Fixes #15140

> http://git.videolan.org/gitweb.cgi/vlc.git/?a=commit;h=d6e400f2505165a1afd10a1bb6925d9772166077
---

 modules/video_chroma/swscale.c |    7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/modules/video_chroma/swscale.c b/modules/video_chroma/swscale.c
index 4f0beb1..f18017e 100644
--- a/modules/video_chroma/swscale.c
+++ b/modules/video_chroma/swscale.c
@@ -661,8 +661,13 @@ static picture_t *Filter( filter_t *p_filter, picture_t *p_pic )
     else if( p_sys->b_copy )
         SwapUV( p_dst, p_src );
     else
+    {
+        /* Even if alpha is unused, swscale expects the pointer to be set */
+        const int n_planes = !p_sys->ctxA && (p_src->i_planes == 4 ||
+                             p_dst->i_planes == 4) ? 4 : 3;
         Convert( p_filter, p_sys->ctx, p_dst, p_src, p_fmti->i_visible_height,
-                 3, p_sys->b_swap_uvi, p_sys->b_swap_uvo );
+                 n_planes, p_sys->b_swap_uvi, p_sys->b_swap_uvo );
+    }
     if( p_sys->ctxA )
     {
         /* We extract the A plane to rescale it, and then we reinject it. */