[vlc-commits] [Git][videolan/vlc][3.0.x] ffmpeg: fix DXVA2/D3D11 crash on bogus references

[Hugo Beauzée-Luyssen (@chouquette)]

Hugo Beauzée-Luyssen pushed to branch 3.0.x at VideoLAN / VLC


Commits:
0d556269 by Steve Lhomme at 2022-03-29T20:08:05+00:00
ffmpeg: fix DXVA2/D3D11 crash on bogus references

Fixes #26738

(cherry picked from commit 1e0b78a970afaabf6af236457cc363d458819787) (edited)
edited:
- on 3.0 the patch needs to be set on FFmpeg only
- the emscripten patch before this one is not on 3.0

Signed-off-by: Steve Lhomme <robux4 at ycbcr.xyz>

- - - - -


2 changed files:

- + contrib/src/ffmpeg/0001-dxva2_hevc-don-t-use-frames-as-reference-if-they-are.patch
- contrib/src/ffmpeg/rules.mak


Changes:

=====================================
contrib/src/ffmpeg/0001-dxva2_hevc-don-t-use-frames-as-reference-if-they-are.patch
=====================================
@@ -0,0 +1,34 @@
+From b2e89bd9c548d899ec2fecfc1aa550e9ddfe9777 Mon Sep 17 00:00:00 2001
+From: Steve Lhomme <robux4 at ycbcr.xyz>
+Date: Wed, 23 Mar 2022 14:52:31 +0100
+Subject: [PATCH] dxva2_hevc: don't use frames as reference if they are not
+ marked as such
+
+Similar to how a frame is considered for referencing for the RefPicList array.
+This will do the same for RefPicSetStCurrBefore, RefPicSetStCurrAfter and
+RefPicSetLtCurr.
+
+Fixes playback of http://www.gbbsoft.pl/!download/!/Film1.mp4
+Ref. VLC issue https://code.videolan.org/videolan/vlc/-/issues/26738
+
+Signed-off-by: Steve Lhomme <robux4 at ycbcr.xyz>
+---
+ libavcodec/dxva2_hevc.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/libavcodec/dxva2_hevc.c b/libavcodec/dxva2_hevc.c
+index 98b3e74bd7..28c0b26733 100644
+--- a/libavcodec/dxva2_hevc.c
++++ b/libavcodec/dxva2_hevc.c
+@@ -249,7 +249,7 @@ static void fill_picture_parameters(const AVCodecContext *avctx, AVDXVAContext *
+             const HEVCFrame *frame = NULL; \
+             while (!frame && j < rpl->nb_refs) \
+                 frame = rpl->ref[j++]; \
+-            if (frame) \
++            if (frame && frame->flags & (HEVC_FRAME_FLAG_LONG_REF | HEVC_FRAME_FLAG_SHORT_REF)) \
+                 pp->ref_list[i] = get_refpic_index(pp, ff_dxva2_get_surface_index(avctx, ctx, frame->frame)); \
+             else \
+                 pp->ref_list[i] = 0xff; \
+-- 
+2.27.0.windows.1
+


=====================================
contrib/src/ffmpeg/rules.mak
=====================================
@@ -240,6 +240,7 @@ ifdef USE_FFMPEG
 	$(APPLY) $(SRC)/ffmpeg/0001-fix-MediaFoundation-compilation-if-WINVER-was-forced.patch
 	$(APPLY) $(SRC)/ffmpeg/0001-bring-back-XP-support.patch
 	$(APPLY) $(SRC)/ffmpeg/0001-avcodec-vp9-Do-not-destroy-uninitialized-mutexes-con.patch
+	$(APPLY) $(SRC)/ffmpeg/0001-dxva2_hevc-don-t-use-frames-as-reference-if-they-are.patch
 endif
 ifdef USE_LIBAV
 	$(APPLY) $(SRC)/ffmpeg/libav_gsm.patch



View it on GitLab: https://code.videolan.org/videolan/vlc/-/commit/0d556269778d0b9d0e9d05cc78b3365e0e74792e

-- 
View it on GitLab: https://code.videolan.org/videolan/vlc/-/commit/0d556269778d0b9d0e9d05cc78b3365e0e74792e
You're receiving this email because of your account on code.videolan.org.


VideoLAN code repository instance