[vlc-commits] d3d11_fmt: detect bogus pitch

[Steve Lhomme]

vlc | branch: master | Steve Lhomme <robUx4 at videolabs.io> | Tue Nov 28 11:19:26 2017 +0100| [676e46d8931a106f9a179f5ca802473f42a39112] | committer: Jean-Baptiste Kempf

d3d11_fmt: detect bogus pitch

Fixes #19162

Signed-off-by: Jean-Baptiste Kempf <jb at videolan.org>

> http://git.videolan.org/gitweb.cgi/vlc.git/?a=commit;h=676e46d8931a106f9a179f5ca802473f42a39112
---

 modules/video_chroma/d3d11_fmt.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/modules/video_chroma/d3d11_fmt.c b/modules/video_chroma/d3d11_fmt.c
index 0553041d7f..2d6e696c20 100644
--- a/modules/video_chroma/d3d11_fmt.c
+++ b/modules/video_chroma/d3d11_fmt.c
@@ -412,6 +412,14 @@ int AllocateTextures( vlc_object_t *obj, d3d11_device_t *d3d_dev,
                      p_chroma_desc->pixel_size * texDesc.Width );
             goto error;
         }
+        if ( mappedResource.RowPitch >=
+             2* (fmt->i_width * p_chroma_desc->p[0].w.num / p_chroma_desc->p[0].w.den * p_chroma_desc->pixel_size) )
+        {
+            msg_Err(obj, "Bogus %4.4s pitch detected. %d vs %d", (const char*)&fmt->i_chroma,
+                    mappedResource.RowPitch,
+                    (fmt->i_width * p_chroma_desc->p[0].w.num / p_chroma_desc->p[0].w.den * p_chroma_desc->pixel_size));
+            goto error;
+        }
     }
 
     if (slicedTexture)