[vlc-commits] codec/avcodec/video: only flush buffer if avcodec_is_open

[Filip Roséen]

vlc | branch: master | Filip Roséen <filip at videolabs.io> | Sun Jul 17 05:54:57 2016 +0200| [57c893667105bb518f9b2b050b4287c2ae80302e] | committer: Rémi Denis-Courmont

codec/avcodec/video: only flush buffer if avcodec_is_open

This patch fixes a crash when trying to invoke avcodec_flush_buffers
on an not-yet-opened AvCodecContext*.

The previous if-condition is faulty because nothing guarantees that
the state of the variable in question reflects whether we are in a
state where p_sys->p_context is valid; instead it has been
replaced by a call to "avcodec_is_open".

refs: #17080

--

"p_sys->p_context" is opened (through avcodec_open2) in "ffmpeg_OpenCodec",
which is not guaranteed to have been called prior to "EndVideoDec" (due to
potential missing data required for codec initialization).

See the implementation of OpenVideoCodec in codec/avcodec/video.c for more
information.

Signed-off-by: Rémi Denis-Courmont <remi at remlab.net>

> http://git.videolan.org/gitweb.cgi/vlc.git/?a=commit;h=57c893667105bb518f9b2b050b4287c2ae80302e
---

 modules/codec/avcodec/video.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/modules/codec/avcodec/video.c b/modules/codec/avcodec/video.c
index a97fec2..0c10f9e 100644
--- a/modules/codec/avcodec/video.c
+++ b/modules/codec/avcodec/video.c
@@ -935,7 +935,7 @@ void EndVideoDec( decoder_t *p_dec )
     post_mt( p_sys );
 
     /* do not flush buffers if codec hasn't been opened (theora/vorbis/VC1) */
-    if( p_sys->p_context->codec )
+    if( avcodec_is_open( p_sys->p_context ) )
         avcodec_flush_buffers( p_sys->p_context );
 
     wait_mt( p_sys );