[vlc-devel] [PATCH] Fixed memory leak when using the mosaic module (ticket #2326).

[Phil Roffe]

From: Phil Roffe and David Grellscheid <philip.roffe at durham.ac.uk>

Fixed by decrementing the i_refcount variable in the correct manner -
and therefore freeing the memory only when the i_refcount successfully
went to 0.

The problem was that i_refcount is an unsigned variable, and was being
decremented twice, once erroneously by mosaic_bridge, and then again in
the picture's original pf_release function. If i_refcount started at 1,
it wrapped to the maximum unsigned value rather than -1, failing the
refcount tests in the pf_release function.

Patch Authors: Phil Roffe and David Grellscheid
---
 modules/stream_out/mosaic_bridge.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/modules/stream_out/mosaic_bridge.c
b/modules/stream_out/mosaic_bridge.c
index e7a8d85..4a8e638 100644
--- a/modules/stream_out/mosaic_bridge.c
+++ b/modules/stream_out/mosaic_bridge.c
@@ -80,8 +80,6 @@ static void ReleasePicture( picture_t *p_pic )
 {
     assert( p_pic );

-    if( --p_pic->i_refcount > 0 )
-        return;

     if( p_pic->p_sys )
     {
@@ -91,9 +89,12 @@ static void ReleasePicture( picture_t *p_pic )
     }
     else
     {
-        free( p_pic->p_q );
-        free( p_pic->p_data_orig );
-        free( p_pic );
+        if( --p_pic->i_refcount == 0 )
+        {
+            free( p_pic->p_q );
+            free( p_pic->p_data_orig );
+            free( p_pic );
+        }
     }
 }