[vlc-commits] mmal/deinterlace: Avoid overflows on buffer calculation

[Julian Scheel]

vlc | branch: master | Julian Scheel <julian at jusst.de> | Wed Jun  3 09:54:37 2015 +0200| [4e8c81f922d938a0d1089571175ed4388ac047ce] | committer: Jean-Baptiste Kempf

mmal/deinterlace: Avoid overflows on buffer calculation

Use signed variables for calculating the buffers to be sent to the component
as negative values may occur.

Signed-off-by: Julian Scheel <julian at jusst.de>
Signed-off-by: Jean-Baptiste Kempf <jb at videolan.org>

> http://git.videolan.org/gitweb.cgi/vlc.git/?a=commit;h=4e8c81f922d938a0d1089571175ed4388ac047ce
---

 modules/hw/mmal/deinterlace.c |   10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/modules/hw/mmal/deinterlace.c b/modules/hw/mmal/deinterlace.c
index e6f6264..f7e2a2c 100644
--- a/modules/hw/mmal/deinterlace.c
+++ b/modules/hw/mmal/deinterlace.c
@@ -331,10 +331,12 @@ static void fill_output_port(filter_t *filter)
 {
     filter_sys_t *sys = filter->p_sys;
     /* allow at least 2 buffers in transit */
-    unsigned max_buffers_in_transit = __MAX(sys->output->buffer_num, MIN_NUM_BUFFERS_IN_TRANSIT);
-    unsigned buffers_available = max_buffers_in_transit - atomic_load(&sys->output_in_transit);
-    unsigned buffers_to_send = max_buffers_in_transit - sys->output_in_transit;
-    unsigned i;
+    unsigned max_buffers_in_transit = __MAX(2, MIN_NUM_BUFFERS_IN_TRANSIT);
+    int buffers_available = sys->output->buffer_num -
+        atomic_load(&sys->output_in_transit) -
+        mmal_queue_length(sys->filtered_pictures);
+    int buffers_to_send = max_buffers_in_transit - sys->output_in_transit;
+    int i;
 
     if (buffers_to_send > buffers_available)
         buffers_to_send = buffers_available;