[vlc-commits] codec: schroedinger: fix potential buffer overflow.

[Fabian Yamaguchi]

vlc/vlc-2.1 | branch: master | Fabian Yamaguchi <fyamagu at gwdg.de> | Fri Dec  5 15:18:22 2014 +0100| [a99d9cc2fede1a18140b43ffb59cde37cef7478d] | committer: Jean-Baptiste Kempf

codec: schroedinger: fix potential buffer overflow.

The variable len is a raw 32 bit value read using GetDWBE. If this
value is larger than UINT32_MAX - sizeof(eos), this will cause an
integer overflow in the subsequent call to malloc, and finally a
buffer overflow when calling memcpy. We fix this by checking len
accordingly.

Signed-off-by: Jean-Baptiste Kempf <jb at videolan.org>
(cherry picked from commit 9bb0353a5c63a7f8c6fc853faa3df4b4df1f5eb5)
Signed-off-by: Jean-Baptiste Kempf <jb at videolan.org>

> http://git.videolan.org/gitweb.cgi/vlc/vlc-2.1.git/?a=commit;h=a99d9cc2fede1a18140b43ffb59cde37cef7478d
---

 modules/codec/schroedinger.c |    4 ++++
 1 file changed, 4 insertions(+)

diff --git a/modules/codec/schroedinger.c b/modules/codec/schroedinger.c
index 0c5a7e7..93b72e1 100644
--- a/modules/codec/schroedinger.c
+++ b/modules/codec/schroedinger.c
@@ -1548,6 +1548,10 @@ static block_t *Encode( encoder_t *p_enc, picture_t *p_pic )
                      * is appended to the sequence header to allow guard
                      * against poor streaming servers */
                     /* XXX, should this be done using the packetizer ? */
+
+                    if( len > UINT32_MAX - sizeof( eos ) )
+                        return NULL;
+
                     p_enc->fmt_out.p_extra = malloc( len + sizeof( eos ) );
                     if( !p_enc->fmt_out.p_extra )
                         return NULL;