[vlc-commits] direct3d11: avoid crashing later in the decoder

[Steve Lhomme]

vlc | branch: master | Steve Lhomme <robUx4 at videolabs.io> | Thu Oct 20 15:17:11 2016 +0200| [57ff42122f34fee45d78937467c0a5285cabb911] | committer: Jean-Baptiste Kempf

direct3d11: avoid crashing later in the decoder

ID3D11VideoDevice::CreateVideoDecoderOutputView() crashes when the texture has
more than 30 slices. Luckily we never need more than that.

Signed-off-by: Jean-Baptiste Kempf <jb at videolan.org>

> http://git.videolan.org/gitweb.cgi/vlc.git/?a=commit;h=57ff42122f34fee45d78937467c0a5285cabb911
---

 modules/video_output/win32/direct3d11.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/modules/video_output/win32/direct3d11.c b/modules/video_output/win32/direct3d11.c
index ad9e80c..23703b9 100644
--- a/modules/video_output/win32/direct3d11.c
+++ b/modules/video_output/win32/direct3d11.c
@@ -552,6 +552,11 @@ static picture_pool_t *Pool(vout_display_t *vd, unsigned pool_size)
     if ( vd->sys->pool != NULL )
         return vd->sys->pool;
 
+    if (pool_size > 30) {
+        msg_Err(vd, "Avoid crashing when using ID3D11VideoDecoderOutputView with too many slices");
+        return NULL;
+    }
+
 #ifdef HAVE_ID3D11VIDEODECODER
     picture_t**       pictures = NULL;
     unsigned          picture_count = 0;