[vlc-commits] video_filter: hqdn3d: reject computed height <= 0 (cid #718140)

[Francois Cartegnie]

vlc | branch: master | Francois Cartegnie <fcvlcdev at free.fr> | Thu Dec 31 16:39:04 2015 +0100| [6c5c07b96b1a349fe76473a84e8b561522bcea4d] | committer: Francois Cartegnie

video_filter: hqdn3d: reject computed height <= 0 (cid #718140)

Otherwise can make use of malloc(0) buffers

> http://git.videolan.org/gitweb.cgi/vlc.git/?a=commit;h=6c5c07b96b1a349fe76473a84e8b561522bcea4d
---

 modules/video_filter/hqdn3d.c |    6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/modules/video_filter/hqdn3d.c b/modules/video_filter/hqdn3d.c
index 64ea5e4..7b63bde 100644
--- a/modules/video_filter/hqdn3d.c
+++ b/modules/video_filter/hqdn3d.c
@@ -136,6 +136,12 @@ static int Open(vlc_object_t *this)
         sys->w[i] = fmt_in->i_width  * chroma->p[i].w.num / chroma->p[i].w.den;
         if (sys->w[i] > wmax) wmax = sys->w[i];
         sys->h[i] = fmt_out->i_height * chroma->p[i].h.num / chroma->p[i].h.den;
+
+        if(sys->w[i] <= 0 || sys->h[i] <= 0)
+        {
+            free(sys);
+            return VLC_EGENERIC;
+        }
     }
     cfg->Line = malloc(wmax*sizeof(unsigned int));
     if (!cfg->Line) {