[vlc-commits] videotoolbox: fix frame leaks when closing decoder

[Thomas Guillem]

vlc | branch: master | Thomas Guillem <thomas at gllm.fr> | Fri Nov 24 14:28:59 2017 +0100| [bee2a18aa920b7a7e19b19a3f78124ff07e6e091] | committer: Thomas Guillem

videotoolbox: fix frame leaks when closing decoder

Set b_vt_flush to true to prevent the asynchronous callback to enqueue more
frames.

This fixes a regression from f75db2e7ef22b2bb9bc6e2ade65793163ebea4e5.

> http://git.videolan.org/gitweb.cgi/vlc.git/?a=commit;h=bee2a18aa920b7a7e19b19a3f78124ff07e6e091
---

 modules/codec/videotoolbox.m | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/modules/codec/videotoolbox.m b/modules/codec/videotoolbox.m
index a8d88343db..dd246e1bb1 100644
--- a/modules/codec/videotoolbox.m
+++ b/modules/codec/videotoolbox.m
@@ -1739,6 +1739,7 @@ static void Drain(decoder_t *p_dec, bool flush)
 
     /* draining: return last pictures of the reordered queue */
     vlc_mutex_lock(&p_sys->lock);
+    p_sys->b_vt_flush = true;
     DrainDPBLocked(p_dec, flush);
     vlc_mutex_unlock(&p_sys->lock);
 
@@ -1746,6 +1747,7 @@ static void Drain(decoder_t *p_dec, bool flush)
         VTDecompressionSessionWaitForAsynchronousFrames(p_sys->session);
 
     vlc_mutex_lock(&p_sys->lock);
+    assert(RemoveOneFrameFromDPB(p_sys) == NULL);
     p_sys->b_vt_flush = false;
     p_sys->b_vt_feed = false;
     vlc_mutex_unlock(&p_sys->lock);