[vlc-commits] avcodec: update the output format when the decoder requests one

[Steve Lhomme]

vlc/vlc-3.0 | branch: master | Steve Lhomme <robux4 at ycbcr.xyz> | Tue Apr 16 13:22:57 2019 +0200| [4a5791a0093c763dff613c77d09fe1d0d5b9100d] | committer: Steve Lhomme

avcodec: update the output format when the decoder requests one

We may reuse the decoder because the decoder size matches but we still need to
send other changes (like SAR) upstream otherwise we may never notice.

If the vout update fails we don't reuse the decoder and try to use a new one.

Fixes #21329

> http://git.videolan.org/gitweb.cgi/vlc/vlc-3.0.git/?a=commit;h=4a5791a0093c763dff613c77d09fe1d0d5b9100d
---

 NEWS                          | 1 +
 modules/codec/avcodec/video.c | 7 +++++--
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/NEWS b/NEWS
index 241c6970a6..bfcb4d357a 100644
--- a/NEWS
+++ b/NEWS
@@ -10,6 +10,7 @@ Audio output:
 Video Output:
  * Fix 12 bits sources playback with Direct3D11
  * Fix crash on iOS
+ * Fix midstream aspect-ratio changes when Windows hardware decoding is on
 
 Stream Output:
  * Improve Chromecast support with new ChromeCast apps
diff --git a/modules/codec/avcodec/video.c b/modules/codec/avcodec/video.c
index 8cb4003561..097e7cb11a 100644
--- a/modules/codec/avcodec/video.c
+++ b/modules/codec/avcodec/video.c
@@ -1557,8 +1557,11 @@ static enum PixelFormat ffmpeg_GetFormat( AVCodecContext *p_context,
      for (size_t i = 0; pi_fmt[i] != AV_PIX_FMT_NONE; i++)
         if (pi_fmt[i] == p_sys->pix_fmt)
         {
-            msg_Dbg(p_dec, "reusing decoder output format %d", pi_fmt[i]);
-            return p_sys->pix_fmt;
+            if (lavc_UpdateVideoFormat(p_dec, p_context, p_sys->pix_fmt, swfmt) == 0)
+            {
+                msg_Dbg(p_dec, "reusing decoder output format %d", pi_fmt[i]);
+                return p_sys->pix_fmt;
+            }
         }
 
 no_reuse: