[vlc-commits] [Git][videolan/vlc][3.0.x] 12 commits: packetizer:av1: fix debug formatting
Steve Lhomme (@robUx4)
gitlab at videolan.org
Tue May 9 07:59:45 UTC 2023
Steve Lhomme pushed to branch 3.0.x at VideoLAN / VLC
Commits:
86e8d3bf by Steve Lhomme at 2023-05-09T07:46:56+00:00
packetizer:av1: fix debug formatting
(cherry picked from commit db1a38ff9e5eb3c8237e7fb42e71ccdc8c990180)
Signed-off-by: Steve Lhomme <robux4 at ycbcr.xyz>
- - - - -
6f8f2562 by Steve Lhomme at 2023-05-09T07:46:56+00:00
packetizer:av1: remove always true test
(cherry picked from commit fe5c91cf4f5b57fdf4619dfaa34f14dfaec32f66)
Signed-off-by: Steve Lhomme <robux4 at ycbcr.xyz>
- - - - -
33adb05c by Steve Lhomme at 2023-05-09T07:46:56+00:00
packetizer:av1: detect sequence header changes
When the sequence header changes we need to know it has changed so the output
values can be reset.
In particular the visible size was not reset since it was set initially by the
demuxer. The first sequence header received is still not used to overwrite the
visible size after this patch.
The extra data were not reset even though they contain the sequence header.
Because there changes were not detected the new output format is never
forwarded.
Fixes #26811
(cherry picked from commit e65c74fdab537df17bfeadf554b1c1d2873120ea) (rebased)
rebased:
* color_range is b_color_range_full on 3.0
Signed-off-by: Steve Lhomme <robux4 at ycbcr.xyz>
- - - - -
80ed5157 by Steve Lhomme at 2023-05-09T07:46:56+00:00
packetizer: av1: provide the OBU chroma
The VLC chroma contains both the subsampling and the bitdepth.
VLC_CODEC_GREY* is used for monochrome.
(cherry picked from commit bd4e454969fd69658d78eca70ecd51abfaaf0060) (rebased)
rebased:
* 3.0 doesn't have COLOR_RANGE_LIMITED so we use 0/1.
Signed-off-by: Steve Lhomme <robux4 at ycbcr.xyz>
- - - - -
f2d4e768 by Steve Lhomme at 2023-05-09T07:46:56+00:00
packetizer: av1: always set subsampling_x/y
Especially since it's mostly 1 (4:2:0) and we were supposed to read the
chroma_sample_position in that case.
(cherry picked from commit b1e684c77a725884d315989a5341d0522d16d1db) (rebased)
rebased:
* COLOR_RANGE_LIMITED doesn't exist in 3.0 so we use 0/1
Signed-off-by: Steve Lhomme <robux4 at ycbcr.xyz>
- - - - -
cac55752 by Steve Lhomme at 2023-05-09T07:46:56+00:00
packetizer:av1: add a function to compare sequence headers
A change in a sequence header means the stream has changed.
(cherry picked from commit d675af28297164e3c67bee00d4bb0b507e72adee)
Signed-off-by: Steve Lhomme <robux4 at ycbcr.xyz>
- - - - -
34d5ba17 by Steve Lhomme at 2023-05-09T07:46:56+00:00
avformat: demux: consider AV1 with no extra data as not packetized
In may come in packetized form but the extra data are not given (from raw OBU
files).
(cherry picked from commit 8a60c1c4ad19ca896cd88efcadc37918c0a7c6b0) (rebased)
rebased:
* the code around is slightly different
Signed-off-by: Steve Lhomme <robux4 at ycbcr.xyz>
- - - - -
d7bb8707 by Steve Lhomme at 2023-05-09T07:46:56+00:00
directx_va: allow AV1 decoding
(cherry picked from commit 50aeffdf10f99f5ef351d08a73e59449c994700c) (edited)
edited:
* on 4.0 there's also the chroma subsampling information per format.
Signed-off-by: Steve Lhomme <robux4 at ycbcr.xyz>
- - - - -
8b6eab7d by Steve Lhomme at 2023-05-09T07:46:56+00:00
avcodec: split the InitVideoDec function in 2 parts
No functional changes.
Also move the profile initialization with the p_sys init.
(cherry picked from commit aad2bdfd9ef4f168036911652af10430f3196d5b) (rebased)
rebase:
* the code around the code move is slightly different
Signed-off-by: Steve Lhomme <robux4 at ycbcr.xyz>
- - - - -
a2350b37 by Steve Lhomme at 2023-05-09T07:46:56+00:00
avcodec: move the code to open a VA in a function
Now the (un)locking is optional.
(cherry picked from commit 4416251e1bb5d5e16021a053056fddf3fb7fce4c) (edited)
edited:
* 3.0 uses a vlc_sem_t instead of a vlc_mutex_t
* 3.0 uses vlc_va_GetChroma() as there is no vlc_va_MightDecode()
* 3.0 doesn't use a decoder device or video context but uses a test picture
from the vout pool
* 3.0 uses more version checks for AV_PIX_FMT values
Signed-off-by: Steve Lhomme <robux4 at ycbcr.xyz>
- - - - -
fc7816c9 by Steve Lhomme at 2023-05-09T07:46:56+00:00
avcodec: add a flag to only allow hardware decoding for the decoder
Only use 1 decoder thread when allowing only hardware decoders.
Restarting the decoder silently is not allowed. Either we can reuse the va or
we need to reload the decoder outside of lavc.
(cherry picked from commit 1effe16f9ff83042843b2fdda6bc7abe68cb7103) (edited)
edited:
* 3.0 doesn't have a defaultfmt but uses p_sys->pix_fmt as it's missing
906ffbaa631f54714ca55731b99f20d18100938e
* 3.0 calls vlc_va_Delete() with a hwaccel_context
Signed-off-by: Steve Lhomme <robux4 at ycbcr.xyz>
- - - - -
99b71d93 by Steve Lhomme at 2023-05-09T07:46:56+00:00
avcodec: add a hardware-only lavc AV1 video decoder with higher priority
The priority is higher than dav1d so it can handle hardware decoding even if
dav1d is present.
This version of lavc should only allow decoding if the decoder has a profile
that can be decoded by the hardware, otherwise it should let other decoders try
to do the decoding (dav1d, lavc with current priority, etc).
(cherry picked from commit cd4d32f71b27c439a44ff37abf2718dc064f2ff1) (edited)
edited:
* 3.0 doesn't include libavutil headers for metadata
* in 3.0 p_dec->fmt_in is not a pointer
* 3.0 doesn't have the meson file modifications
Signed-off-by: Steve Lhomme <robux4 at ycbcr.xyz>
- - - - -
9 changed files:
- modules/codec/Makefile.am
- modules/codec/avcodec/avcodec.c
- modules/codec/avcodec/avcodec.h
- modules/codec/avcodec/directx_va.c
- modules/codec/avcodec/video.c
- modules/demux/avformat/demux.c
- modules/packetizer/av1.c
- modules/packetizer/av1_obu.c
- modules/packetizer/av1_obu.h
Changes:
=====================================
modules/codec/Makefile.am
=====================================
@@ -373,7 +373,8 @@ libavcodec_plugin_la_SOURCES = \
codec/avcodec/subtitle.c \
codec/avcodec/audio.c \
codec/avcodec/va.c codec/avcodec/va.h \
- codec/avcodec/avcodec.c codec/avcodec/avcodec.h
+ codec/avcodec/avcodec.c codec/avcodec/avcodec.h \
+ packetizer/av1_obu.c packetizer/av1_obu.h packetizer/av1.h
if ENABLE_SOUT
libavcodec_plugin_la_SOURCES += codec/avcodec/encoder.c
endif
=====================================
modules/codec/avcodec/avcodec.c
=====================================
@@ -85,6 +85,11 @@ vlc_module_begin ()
set_capability("video decoder", 70)
set_callbacks(InitVideoDec, EndVideoDec)
+ add_submodule()
+ add_shortcut("ffmpeghw")
+ set_capability("video decoder", 10001)
+ set_callbacks(InitVideoHwDec, EndVideoDec)
+
add_submodule()
add_shortcut("ffmpeg")
set_capability("audio decoder", 70)
=====================================
modules/codec/avcodec/avcodec.h
=====================================
@@ -36,6 +36,7 @@ void EndVideoEnc( vlc_object_t * );
/* Video Decoder */
int InitVideoDec( vlc_object_t * );
+int InitVideoHwDec( vlc_object_t * );
void EndVideoDec( vlc_object_t * );
/* Audio Decoder */
=====================================
modules/codec/avcodec/directx_va.c
=====================================
@@ -65,6 +65,9 @@ static const int PROF_HEVC_MAIN10[] = { FF_PROFILE_HEVC_MAIN,
static const int PROF_VP9_MAIN[] = { FF_PROFILE_VP9_0, FF_PROFILE_UNKNOWN };
static const int PROF_VP9_10[] = { FF_PROFILE_VP9_2, FF_PROFILE_UNKNOWN };
+static const int PROF_AV1_MAIN[] = { FF_PROFILE_AV1_MAIN, FF_PROFILE_UNKNOWN };
+static const int PROF_AV1_HIGH[] = { FF_PROFILE_AV1_HIGH, FF_PROFILE_AV1_MAIN, FF_PROFILE_UNKNOWN };
+
#include <winapifamily.h>
#if defined(WINAPI_FAMILY)
# undef WINAPI_FAMILY
@@ -166,6 +169,14 @@ DEFINE_GUID(DXVA_ModeVP9_VLD_Profile0, 0x463707f8, 0xa1d0, 0x4585,
DEFINE_GUID(DXVA_ModeVP9_VLD_10bit_Profile2, 0xa4c749ef, 0x6ecf, 0x48aa, 0x84, 0x48, 0x50, 0xa7, 0xa1, 0x16, 0x5f, 0xf7);
DEFINE_GUID(DXVA_ModeVP9_VLD_Intel, 0x76988a52, 0xdf13, 0x419a, 0x8e, 0x64, 0xff, 0xcf, 0x4a, 0x33, 0x6c, 0xf5);
+#ifndef _DIRECTX_AV1_VA_
+DEFINE_GUID(DXVA_ModeAV1_VLD_Profile0, 0xb8be4ccb, 0xcf53, 0x46ba, 0x8d, 0x59, 0xd6, 0xb8, 0xa6, 0xda, 0x5d, 0x2a);
+DEFINE_GUID(DXVA_ModeAV1_VLD_Profile1, 0x6936ff0f, 0x45b1, 0x4163, 0x9c, 0xc1, 0x64, 0x6e, 0xf6, 0x94, 0x61, 0x08);
+DEFINE_GUID(DXVA_ModeAV1_VLD_Profile2, 0x0c5f2aa1, 0xe541, 0x4089, 0xbb, 0x7b, 0x98, 0x11, 0x0a, 0x19, 0xd7, 0xc8);
+DEFINE_GUID(DXVA_ModeAV1_VLD_12bit_Profile2, 0x17127009, 0xa00f, 0x4ce1, 0x99, 0x4e, 0xbf, 0x40, 0x81, 0xf6, 0xf3, 0xf0);
+DEFINE_GUID(DXVA_ModeAV1_VLD_12bit_Profile2_420, 0x2d80bed6, 0x9cac, 0x4835, 0x9e, 0x91, 0x32, 0x7b, 0xbc, 0x4f, 0x9e, 0xe8);
+#endif
+
typedef struct {
const char *name;
const GUID *guid;
@@ -272,6 +283,19 @@ static const directx_va_mode_t DXVA_MODES[] = {
#endif
{ "VP9 profile Intel", &DXVA_ModeVP9_VLD_Intel, 8, 0, NULL },
+ /* AV1 */
+#if LIBAVCODEC_VERSION_INT >= AV_VERSION_INT( 58, 112, 103 ) && LIBAVCODEC_VERSION_MICRO >= 100
+ { "AV1 Main profile 8", &DXVA_ModeAV1_VLD_Profile0, 8, AV_CODEC_ID_AV1, PROF_AV1_MAIN },
+ { "AV1 Main profile 10", &DXVA_ModeAV1_VLD_Profile0, 10, AV_CODEC_ID_AV1, PROF_AV1_MAIN },
+ { "AV1 High profile 8", &DXVA_ModeAV1_VLD_Profile1, 8, AV_CODEC_ID_AV1, PROF_AV1_HIGH },
+ { "AV1 High profile 10", &DXVA_ModeAV1_VLD_Profile1, 10, AV_CODEC_ID_AV1, PROF_AV1_HIGH },
+#else
+ { "AV1 Main profile 8", &DXVA_ModeAV1_VLD_Profile0, 8, 0, NULL },
+ { "AV1 Main profile 10", &DXVA_ModeAV1_VLD_Profile0, 10, 0, NULL },
+ { "AV1 High profile 8", &DXVA_ModeAV1_VLD_Profile1, 8, 0, NULL },
+ { "AV1 High profile 10", &DXVA_ModeAV1_VLD_Profile1, 10, 0, NULL },
+#endif
+
{ NULL, NULL, 0, 0, NULL }
};
@@ -330,6 +354,10 @@ int directx_va_Setup(vlc_va_t *va, directx_sys_t *dx_sys,
case AV_CODEC_ID_VP9:
surface_count += 4;
break;
+ case AV_CODEC_ID_AV1:
+ surface_alignment = 128;
+ surface_count += 8 + 1;
+ break;
default:
surface_count += 2;
}
=====================================
modules/codec/avcodec/video.c
=====================================
@@ -46,6 +46,8 @@
#include "avcodec.h"
#include "va.h"
+#include "../../packetizer/av1_obu.h"
+#include "../../packetizer/av1.h"
#include "../codec/cc.h"
/*****************************************************************************
@@ -66,6 +68,7 @@ struct decoder_sys_t
bool b_hurry_up;
bool b_show_corrupted;
bool b_from_preroll;
+ bool b_hardware_only;
enum AVDiscard i_skip_frame;
/* how many decoded frames are late */
@@ -412,13 +415,14 @@ static int OpenVideoCodec( decoder_t *p_dec )
ctx->width = p_dec->fmt_in.video.i_visible_width;
ctx->height = p_dec->fmt_in.video.i_visible_height;
- ctx->coded_width = p_dec->fmt_in.video.i_width;
- ctx->coded_height = p_dec->fmt_in.video.i_height;
+ if (!ctx->coded_width || !ctx->coded_height)
+ {
+ ctx->coded_width = p_dec->fmt_in.video.i_width;
+ ctx->coded_height = p_dec->fmt_in.video.i_height;
+ }
ctx->bits_per_coded_sample = p_dec->fmt_in.video.i_bits_per_pixel;
p_sys->pix_fmt = AV_PIX_FMT_NONE;
- p_sys->profile = -1;
- p_sys->level = -1;
cc_Init( &p_sys->cc );
set_video_color_settings( &p_dec->fmt_in.video, ctx );
@@ -458,33 +462,13 @@ static int OpenVideoCodec( decoder_t *p_dec )
return 0;
}
-/*****************************************************************************
- * InitVideo: initialize the video decoder
- *****************************************************************************
- * the ffmpeg codec will be opened, some memory allocated. The vout is not yet
- * opened (done after the first decoded frame).
- *****************************************************************************/
-int InitVideoDec( vlc_object_t *obj )
+static int InitVideoDecCommon( decoder_t *p_dec )
{
- decoder_t *p_dec = (decoder_t *)obj;
- const AVCodec *p_codec;
- AVCodecContext *p_context = ffmpeg_AllocContext( p_dec, &p_codec );
- if( p_context == NULL )
- return VLC_EGENERIC;
-
+ decoder_sys_t *p_sys = p_dec->p_sys;
+ AVCodecContext *p_context = p_sys->p_context;
+ const AVCodec *p_codec = p_sys->p_codec;
int i_val;
- /* Allocate the memory needed to store the decoder's structure */
- decoder_sys_t *p_sys = calloc( 1, sizeof(*p_sys) );
- if( unlikely(p_sys == NULL) )
- {
- avcodec_free_context( &p_context );
- return VLC_ENOMEM;
- }
-
- p_dec->p_sys = p_sys;
- p_sys->p_context = p_context;
- p_sys->p_codec = p_codec;
p_sys->p_va = NULL;
vlc_sem_init( &p_sys->sem_mt, 0 );
@@ -556,7 +540,7 @@ int InitVideoDec( vlc_object_t *obj )
p_context->get_buffer2 = lavc_GetFrame;
p_context->opaque = p_dec;
- int i_thread_count = var_InheritInteger( p_dec, "avcodec-threads" );
+ int i_thread_count = p_sys->b_hardware_only ? 1 : var_InheritInteger( p_dec, "avcodec-threads" );
if( i_thread_count <= 0 )
{
i_thread_count = vlc_GetCPUCount();
@@ -649,6 +633,209 @@ int InitVideoDec( vlc_object_t *obj )
return VLC_SUCCESS;
}
+static int ffmpeg_OpenVa(decoder_t *p_dec, AVCodecContext *p_context,
+ enum AVPixelFormat hwfmt, enum AVPixelFormat swfmt,
+ const AVPixFmtDescriptor *src_desc,
+ vlc_sem_t *open_lock)
+{
+ decoder_sys_t *p_sys = p_dec->p_sys;
+
+ if( hwfmt == AV_PIX_FMT_NONE )
+ return VLC_EGENERIC;
+
+ p_dec->fmt_out.video.i_chroma = vlc_va_GetChroma(hwfmt, swfmt);
+ if (p_dec->fmt_out.video.i_chroma == 0)
+ return VLC_EGENERIC; /* Unknown brand of hardware acceleration */
+ if (p_context->width == 0 || p_context->height == 0)
+ { /* should never happen */
+ msg_Err(p_dec, "unspecified video dimensions");
+ return VLC_EGENERIC;
+ }
+ const AVPixFmtDescriptor *dsc = av_pix_fmt_desc_get(hwfmt);
+ msg_Dbg(p_dec, "trying format %s", dsc ? dsc->name : "unknown");
+ if (lavc_UpdateVideoFormat(p_dec, p_context, hwfmt, swfmt))
+ return VLC_EGENERIC; /* Unsupported brand of hardware acceleration */
+
+ if (open_lock)
+ vlc_sem_post(open_lock);
+
+ picture_t *test_pic = decoder_NewPicture(p_dec);
+ assert(!test_pic || test_pic->format.i_chroma == p_dec->fmt_out.video.i_chroma);
+ vlc_va_t *va = vlc_va_New(VLC_OBJECT(p_dec), p_context, src_desc, hwfmt,
+ &p_dec->fmt_in,
+ test_pic ? test_pic->p_sys : NULL);
+
+ if (open_lock)
+ vlc_sem_wait(open_lock);
+
+ if (test_pic)
+ picture_Release(test_pic);
+ if (va == NULL)
+ {
+ return VLC_EGENERIC; /* Unsupported codec profile or such */
+ }
+
+ if (va->description != NULL)
+ msg_Info(p_dec, "Using %s for hardware decoding", va->description);
+
+ p_sys->p_va = va;
+ p_sys->pix_fmt = hwfmt;
+ p_context->draw_horiz_band = NULL;
+ return VLC_SUCCESS;
+}
+
+static const enum PixelFormat hwfmts[] =
+{
+#ifdef _WIN32
+#if LIBAVUTIL_VERSION_CHECK(54, 13, 1, 24, 100)
+ AV_PIX_FMT_D3D11VA_VLD,
+#endif
+ AV_PIX_FMT_DXVA2_VLD,
+#endif
+ AV_PIX_FMT_VAAPI,
+#if (LIBAVUTIL_VERSION_INT >= AV_VERSION_INT(52, 4, 0))
+ AV_PIX_FMT_VDPAU,
+#endif
+ AV_PIX_FMT_NONE,
+};
+
+int InitVideoHwDec( vlc_object_t *obj )
+{
+ decoder_t *p_dec = container_of(obj, decoder_t, obj);
+
+ if (p_dec->fmt_in.i_codec != VLC_CODEC_AV1)
+ return VLC_EGENERIC;
+
+ decoder_sys_t *p_sys = calloc(1, sizeof(*p_sys));
+ if( unlikely(p_sys == NULL) )
+ return VLC_ENOMEM;
+
+ const AVCodec *p_codec;
+ AVCodecContext *p_context = ffmpeg_AllocContext( p_dec, &p_codec );
+ if( unlikely(p_context == NULL) )
+ {
+ free(p_sys);
+ return VLC_ENOMEM;
+ }
+
+ av1_OBU_sequence_header_t *sequence_hdr = NULL;
+ unsigned w, h;
+
+ if (p_dec->fmt_in.i_extra > 4)
+ {
+ // in ISOBMFF/WebM/Matroska the first 4 bytes are from the AV1CodecConfigurationBox
+ // and then one or more OBU
+ const uint8_t *obu_start = ((const uint8_t*) p_dec->fmt_in.p_extra) + 4;
+ int obu_size = p_dec->fmt_in.i_extra - 4;
+ if (AV1_OBUIsValid(obu_start, obu_size) && AV1_OBUGetType(obu_start) == AV1_OBU_SEQUENCE_HEADER)
+ sequence_hdr = AV1_OBU_parse_sequence_header(obu_start, obu_size);
+ }
+
+ if (sequence_hdr == NULL)
+ goto failed;
+
+ // fill the AVCodecContext with the values from the sequence header
+ // so we can create the expected VA right away:
+ // coded_width, coded_height, framerate, profile and sw_pix_fmt
+
+ vlc_fourcc_t chroma = AV1_get_chroma(sequence_hdr);
+ if (chroma == 0)
+ {
+ AV1_release_sequence_header(sequence_hdr);
+ goto failed;
+ }
+ p_context->sw_pix_fmt = FindFfmpegChroma(chroma);
+
+ if (p_context->sw_pix_fmt == AV_PIX_FMT_NONE)
+ {
+ AV1_release_sequence_header(sequence_hdr);
+ goto failed;
+ }
+
+ AV1_get_frame_max_dimensions(sequence_hdr, &w, &h);
+
+ p_context->coded_width = p_context->width = w;
+ p_context->coded_height = p_context->height = h;
+
+ if (!p_dec->fmt_in.video.i_frame_rate || !p_dec->fmt_in.video.i_frame_rate_base)
+ {
+ unsigned num, den;
+ if (AV1_get_frame_rate(sequence_hdr, &num, &den))
+ {
+ p_context->framerate.num = num;
+ p_context->framerate.den = den;
+ }
+ }
+
+ int tier;
+ AV1_get_profile_level(sequence_hdr, &p_sys->profile, &p_sys->level, &tier);
+
+ AV1_release_sequence_header(sequence_hdr);
+
+ p_dec->p_sys = p_sys;
+ p_sys->p_context = p_context;
+ p_sys->p_codec = p_codec;
+ p_sys->pix_fmt = AV_PIX_FMT_NONE;
+ p_sys->b_hardware_only = true;
+
+ int res = InitVideoDecCommon( p_dec );
+ if (res != VLC_SUCCESS)
+ goto not_usable;
+
+ for( size_t i = 0; hwfmts[i] != AV_PIX_FMT_NONE; i++ )
+ {
+ enum AVPixelFormat hwfmt = hwfmts[i];
+
+ const AVPixFmtDescriptor *dsc = av_pix_fmt_desc_get(hwfmt);
+ if (dsc == NULL)
+ continue;
+
+ if (ffmpeg_OpenVa(p_dec, p_context, hwfmt, p_context->sw_pix_fmt, dsc, NULL) == VLC_SUCCESS)
+ // we have a matching hardware decoder
+ return VLC_SUCCESS;
+ }
+
+not_usable:
+ EndVideoDec(obj);
+ return VLC_EGENERIC;
+failed:
+ avcodec_free_context( &p_context );
+ free(p_sys);
+ return VLC_EGENERIC;
+}
+
+/*****************************************************************************
+ * InitVideo: initialize the video decoder
+ *****************************************************************************
+ * the ffmpeg codec will be opened, some memory allocated. The vout is not yet
+ * opened (done after the first decoded frame).
+ *****************************************************************************/
+int InitVideoDec( vlc_object_t *obj )
+{
+ decoder_t *p_dec = (decoder_t *)obj;
+ const AVCodec *p_codec;
+ AVCodecContext *p_context = ffmpeg_AllocContext( p_dec, &p_codec );
+ if( p_context == NULL )
+ return VLC_EGENERIC;
+
+ /* Allocate the memory needed to store the decoder's structure */
+ decoder_sys_t *p_sys = calloc( 1, sizeof(*p_sys) );
+ if( unlikely(p_sys == NULL) )
+ {
+ avcodec_free_context( &p_context );
+ return VLC_ENOMEM;
+ }
+
+ p_dec->p_sys = p_sys;
+ p_sys->p_context = p_context;
+ p_sys->p_codec = p_codec;
+ p_sys->profile = -1;
+ p_sys->level = -1;
+ p_sys->b_hardware_only = false;
+
+ return InitVideoDecCommon( p_dec );
+}
+
/*****************************************************************************
* Flush:
*****************************************************************************/
@@ -1554,6 +1741,11 @@ static enum PixelFormat ffmpeg_GetFormat( AVCodecContext *p_context,
can_hwaccel = true;
}
+ /* Use the default fmt in priority of any sw fmt if the default fmt is a hw
+ * one */
+ if (p_sys->pix_fmt != AV_PIX_FMT_NONE && !p_sys->b_hardware_only)
+ swfmt = p_sys->pix_fmt;
+
if (p_sys->pix_fmt == AV_PIX_FMT_NONE)
goto no_reuse;
@@ -1594,6 +1786,10 @@ no_reuse:
if (p_sys->p_va != NULL)
{
msg_Err(p_dec, "existing hardware acceleration cannot be reused");
+ // the decoder changes have to be handled outside of lavc so that
+ // switching to a software decoder will not silently decode nothing
+ // (get_format will fail to use AV_PIX_FMT_NONE)
+ assert(!p_sys->b_hardware_only);
vlc_va_Delete(p_sys->p_va, &p_context->hwaccel_context);
p_sys->p_va = NULL;
}
@@ -1616,21 +1812,6 @@ no_reuse:
wait_mt(p_sys);
- static const enum PixelFormat hwfmts[] =
- {
-#ifdef _WIN32
-#if LIBAVUTIL_VERSION_CHECK(54, 13, 1, 24, 100)
- AV_PIX_FMT_D3D11VA_VLD,
-#endif
- AV_PIX_FMT_DXVA2_VLD,
-#endif
- AV_PIX_FMT_VAAPI,
-#if (LIBAVUTIL_VERSION_INT >= AV_VERSION_INT(52, 4, 0))
- AV_PIX_FMT_VDPAU,
-#endif
- AV_PIX_FMT_NONE,
- };
-
const AVPixFmtDescriptor *src_desc = av_pix_fmt_desc_get(swfmt);
for( size_t i = 0; hwfmts[i] != AV_PIX_FMT_NONE; i++ )
@@ -1640,42 +1821,10 @@ no_reuse:
if( hwfmts[i] == pi_fmt[j] )
hwfmt = hwfmts[i];
- if( hwfmt == AV_PIX_FMT_NONE )
+ if (ffmpeg_OpenVa(p_dec, p_context, hwfmt, swfmt, src_desc, &p_sys->sem_mt) != VLC_SUCCESS)
continue;
- p_dec->fmt_out.video.i_chroma = vlc_va_GetChroma(hwfmt, swfmt);
- if (p_dec->fmt_out.video.i_chroma == 0)
- continue; /* Unknown brand of hardware acceleration */
- if (p_context->width == 0 || p_context->height == 0)
- { /* should never happen */
- msg_Err(p_dec, "unspecified video dimensions");
- continue;
- }
- const AVPixFmtDescriptor *dsc = av_pix_fmt_desc_get(hwfmt);
- msg_Dbg(p_dec, "trying format %s", dsc ? dsc->name : "unknown");
- if (lavc_UpdateVideoFormat(p_dec, p_context, hwfmt, swfmt))
- continue; /* Unsupported brand of hardware acceleration */
post_mt(p_sys);
-
- picture_t *test_pic = decoder_NewPicture(p_dec);
- assert(!test_pic || test_pic->format.i_chroma == p_dec->fmt_out.video.i_chroma);
- vlc_va_t *va = vlc_va_New(VLC_OBJECT(p_dec), p_context, src_desc, hwfmt,
- &p_dec->fmt_in,
- test_pic ? test_pic->p_sys : NULL);
- if (test_pic)
- picture_Release(test_pic);
- if (va == NULL)
- {
- wait_mt(p_sys);
- continue; /* Unsupported codec profile or such */
- }
-
- if (va->description != NULL)
- msg_Info(p_dec, "Using %s for hardware decoding", va->description);
-
- p_sys->p_va = va;
- p_sys->pix_fmt = hwfmt;
- p_context->draw_horiz_band = NULL;
return hwfmt;
}
=====================================
modules/demux/avformat/demux.c
=====================================
@@ -663,6 +663,11 @@ int avformat_OpenDemux( vlc_object_t *p_this )
memcpy( es_fmt.p_extra, p_extra, i_extra );
}
}
+ else if ( cp->codec_id == AV_CODEC_ID_AV1 )
+ {
+ // raw AV1, we need a packetizer to detect configuration changes
+ es_fmt.b_packetized = false;
+ }
p_track->p_es = es_out_Add( p_demux->out, &es_fmt );
if( p_track->p_es && (s->disposition & AV_DISPOSITION_DEFAULT) )
=====================================
modules/packetizer/av1.c
=====================================
@@ -52,6 +52,7 @@ struct decoder_sys_t
block_t *p_sequence_header_block;
av1_OBU_sequence_header_t *p_sequence_header;
+ bool b_sequence_header_changed;
struct
{
bool b_has_visible_frame;
@@ -113,7 +114,8 @@ static void UpdateDecoderFormat(decoder_t *p_dec)
unsigned wnum, hden;
AV1_get_frame_max_dimensions(p_sys->p_sequence_header, &wnum, &hden);
if((!p_dec->fmt_in.video.i_visible_height ||
- !p_dec->fmt_in.video.i_visible_width) &&
+ !p_dec->fmt_in.video.i_visible_width ||
+ p_sys->b_sequence_header_changed) &&
(p_dec->fmt_out.video.i_visible_width != wnum ||
p_dec->fmt_out.video.i_visible_width != hden))
{
@@ -150,6 +152,12 @@ static void UpdateDecoderFormat(decoder_t *p_dec)
p_dec->fmt_out.video.b_color_range_full = full;
}
+ if (p_sys->b_sequence_header_changed && p_dec->fmt_out.p_extra)
+ {
+ free(p_dec->fmt_out.p_extra);
+ p_dec->fmt_out.i_extra = 0;
+ }
+
if(!p_dec->fmt_in.i_extra && !p_dec->fmt_out.i_extra)
{
p_dec->fmt_out.i_extra =
@@ -159,6 +167,7 @@ static void UpdateDecoderFormat(decoder_t *p_dec)
(const uint8_t **)&p_sys->p_sequence_header_block->p_buffer,
&p_sys->p_sequence_header_block->i_buffer);
}
+ p_sys->b_sequence_header_changed = false;
}
static block_t * OutputQueues(decoder_t *p_dec, bool b_valid)
@@ -221,7 +230,7 @@ static block_t *GatherAndValidateChain(decoder_t *p_dec, block_t *p_outputchain)
if(p_outputchain)
{
#ifdef DEBUG_AV1_PACKETIZER
- msg_Dbg(p_dec, "TU output %ld", p_outputchain->i_dts);
+ msg_Dbg(p_dec, "TU output %" PRId64, p_outputchain->i_dts);
for(block_t *p = p_outputchain; p; p=p->p_next)
{
enum av1_obu_type_e OBUtype = AV1_OBUGetType(p->p_buffer);
@@ -234,7 +243,7 @@ static block_t *GatherAndValidateChain(decoder_t *p_dec, block_t *p_outputchain)
p_sys->p_sequence_header);
if(p_fh)
{
- msg_Dbg(p_dec,"OBU TYPE %d sz %ld dts %ld type %d %d",
+ msg_Dbg(p_dec,"OBU TYPE %d sz %zu dts %" PRId64 " type %d %d",
OBUtype, p->i_buffer, p->i_dts,
AV1_get_frame_type(p_fh),
AV1_get_frame_visibility(p_fh));
@@ -242,7 +251,7 @@ static block_t *GatherAndValidateChain(decoder_t *p_dec, block_t *p_outputchain)
AV1_release_frame_header(p_fh);
}
}
- else msg_Dbg(p_dec, "OBU TYPE %d sz %ld dts %ld", OBUtype, p->i_buffer, p->i_dts);
+ else msg_Dbg(p_dec, "OBU TYPE %d sz %zu dts %" PRId64, OBUtype, p->i_buffer, p->i_dts);
}
#endif
if(p_outputchain->i_flags & BLOCK_FLAG_DROP)
@@ -286,9 +295,21 @@ static block_t *ParseOBUBlock(decoder_t *p_dec, block_t *p_obu)
p_sys->p_sequence_header_block = block_Duplicate(p_obu);
}
- if(p_sys->p_sequence_header)
- AV1_release_sequence_header(p_sys->p_sequence_header);
- p_sys->p_sequence_header = AV1_OBU_parse_sequence_header(p_obu->p_buffer, p_obu->i_buffer);
+ av1_OBU_sequence_header_t *new_seq_header;
+ new_seq_header = AV1_OBU_parse_sequence_header(p_obu->p_buffer, p_obu->i_buffer);
+ if (likely(new_seq_header))
+ {
+ if (!p_sys->p_sequence_header ||
+ !AV1_sequence_header_equal(p_sys->p_sequence_header, new_seq_header))
+ {
+ if (p_sys->p_sequence_header)
+ {
+ AV1_release_sequence_header(p_sys->p_sequence_header);
+ p_sys->b_sequence_header_changed = true;
+ }
+ p_sys->p_sequence_header = new_seq_header;
+ }
+ }
}
PUSHQ(tu.pre, p_obu);
} break;
@@ -312,7 +333,7 @@ static block_t *ParseOBUBlock(decoder_t *p_dec, block_t *p_obu)
if(p_fh)
{
if((p_sys->i_seen & AV1_OBU_TEMPORAL_DELIMITER) && p_sys->tu.b_has_visible_frame)
- p_output = OutputQueues(p_dec, p_sys->p_sequence_header != NULL);
+ p_output = OutputQueues(p_dec, true);
switch(AV1_get_frame_type(p_fh))
{
@@ -380,6 +401,7 @@ static void PacketizeFlush(decoder_t *p_dec)
{
AV1_release_sequence_header(p_sys->p_sequence_header);
p_sys->p_sequence_header = NULL;
+ p_sys->b_sequence_header_changed = true;
}
if(p_sys->p_sequence_header_block)
{
@@ -528,6 +550,7 @@ static int Open(vlc_object_t *p_this)
INITQ(obus);
p_sys->p_sequence_header_block = NULL;
p_sys->p_sequence_header = NULL;
+ p_sys->b_sequence_header_changed = false;
p_sys->tu.b_has_visible_frame = false;
p_sys->tu.dts = VLC_TICK_INVALID;
p_sys->tu.pts = VLC_TICK_INVALID;
=====================================
modules/packetizer/av1_obu.c
=====================================
@@ -162,6 +162,8 @@ struct av1_color_config_s
obu_u1_t subsampling_y;
obu_u2_t chroma_sample_position;
obu_u1_t separate_uv_delta_q;
+
+ vlc_fourcc_t i_chroma;
};
static bool av1_parse_color_config(bs_t *p_bs,
@@ -195,12 +197,18 @@ static bool av1_parse_color_config(bs_t *p_bs,
if(p_cc->mono_chrome)
{
p_cc->color_range = bs_read1(p_bs);
+ p_cc->i_chroma = VLC_CODEC_GREY;
+ p_cc->subsampling_x = 1;
+ p_cc->subsampling_y = 1;
}
else if( p_cc->color_primaries == 1 &&
p_cc->transfer_characteristics == 13 &&
p_cc->matrix_coefficients == 0 )
{
p_cc->color_range = 1;
+ p_cc->i_chroma = VLC_CODEC_I444;
+ p_cc->subsampling_x = 0;
+ p_cc->subsampling_y = 0;
}
else
{
@@ -210,14 +218,31 @@ static bool av1_parse_color_config(bs_t *p_bs,
if(BitDepth == 12)
{
p_cc->subsampling_x = bs_read1(p_bs);
- if(p_cc->subsampling_x)
- p_cc->subsampling_y = bs_read1(p_bs);
+ p_cc->subsampling_y = p_cc->subsampling_x ? bs_read1(p_bs) : 0;
}
else
{
p_cc->subsampling_x = 1;
+ p_cc->subsampling_y = 0;
}
+ p_cc->i_chroma = p_cc->subsampling_x ?
+ p_cc->subsampling_y ? VLC_CODEC_I420 :
+ VLC_CODEC_I422 :
+ VLC_CODEC_I444;
+ }
+ else if(seq_profile == 1)
+ {
+ p_cc->i_chroma = VLC_CODEC_I444;
+ p_cc->subsampling_x = 0;
+ p_cc->subsampling_y = 0;
+ }
+ else
+ {
+ p_cc->i_chroma = VLC_CODEC_I420;
+ p_cc->subsampling_x = 1;
+ p_cc->subsampling_y = 1;
}
+
if(p_cc->subsampling_x && p_cc->subsampling_y)
p_cc->chroma_sample_position = bs_read(p_bs, 2);
}
@@ -546,6 +571,54 @@ bool AV1_get_colorimetry(const av1_OBU_sequence_header_t *p_seq,
return true;
}
+vlc_fourcc_t AV1_get_chroma(const av1_OBU_sequence_header_t *p_seq)
+{
+ switch (p_seq->color_config.i_chroma)
+ {
+ case VLC_CODEC_GREY:
+ switch (p_seq->color_config.high_bitdepth + p_seq->color_config.twelve_bit)
+ {
+ case 0: return VLC_CODEC_GREY;
+ case 1: return VLC_CODEC_GREY_10L;
+ case 2: return VLC_CODEC_GREY_12L;
+ default:
+ vlc_assert_unreachable();
+ }
+ break;
+ case VLC_CODEC_I420:
+ switch (p_seq->color_config.high_bitdepth + p_seq->color_config.twelve_bit)
+ {
+ case 0: return VLC_CODEC_I420;
+ case 1: return VLC_CODEC_I420_10L;
+ case 2: return VLC_CODEC_I420_12L;
+ default:
+ vlc_assert_unreachable();
+ }
+ break;
+ case VLC_CODEC_I422:
+ switch (p_seq->color_config.high_bitdepth + p_seq->color_config.twelve_bit)
+ {
+ case 0: return VLC_CODEC_I422;
+ case 1: return VLC_CODEC_I422_10L;
+ case 2: return VLC_CODEC_I422_12L;
+ default:
+ vlc_assert_unreachable();
+ }
+ break;
+ case VLC_CODEC_I444:
+ switch (p_seq->color_config.high_bitdepth + p_seq->color_config.twelve_bit)
+ {
+ case 0: return VLC_CODEC_I444;
+ case 1: return VLC_CODEC_I444_10L;
+ case 2: return VLC_CODEC_I444_12L;
+ default:
+ vlc_assert_unreachable();
+ }
+ default:
+ vlc_assert_unreachable();
+ }
+}
+
size_t AV1_create_DecoderConfigurationRecord(uint8_t **pp_buffer,
const av1_OBU_sequence_header_t *p_seq,
size_t i_obu, const uint8_t *p_obus[],
@@ -586,3 +659,88 @@ size_t AV1_create_DecoderConfigurationRecord(uint8_t **pp_buffer,
*pp_buffer = p_buffer;
return i_buffer;
}
+
+bool AV1_sequence_header_equal(const av1_OBU_sequence_header_t *seq1,const av1_OBU_sequence_header_t *seq2)
+{
+#define DIFF(field) \
+ seq1->field != seq2->field ||
+
+ if (
+ DIFF(obu_header.obu_type)
+ DIFF(obu_header.temporal_id)
+ DIFF(obu_header.spatial_id)
+ DIFF(seq_profile)
+ DIFF(still_picture)
+ DIFF(reduced_still_picture_header)
+ DIFF(timing_info_present_flag)
+ DIFF(timing_info.num_units_in_display_tick)
+ DIFF(timing_info.time_scale)
+ DIFF(timing_info.equal_picture_interval)
+ DIFF(timing_info.num_ticks_per_picture_minus_1)
+ DIFF(decoder_model_info_present_flag)
+ DIFF(decoder_model_info.buffer_delay_length_minus_1)
+ DIFF(decoder_model_info.num_units_in_decoding_tick)
+ DIFF(decoder_model_info.buffer_removal_time_length_minus_1)
+ DIFF(decoder_model_info.frame_presentation_time_length_minus_1)
+ DIFF(initial_display_delay_present_flag)
+ DIFF(operating_points_cnt_minus_1)
+ DIFF(max_frame_width_minus_1)
+ DIFF(max_frame_height_minus_1)
+ DIFF(frame_id_numbers_present_flag)
+ DIFF(delta_frame_id_length_minus_2)
+ DIFF(additional_frame_id_length_minus_1)
+ DIFF(use_128x128_superblock)
+ DIFF(enable_filter_intra)
+ DIFF(enable_intra_edge_filter)
+
+ DIFF(enable_interintra_compound)
+ DIFF(enable_masked_compound)
+ DIFF(enable_warped_motion)
+ DIFF(enable_dual_filter)
+ DIFF(enable_order_hint)
+ DIFF(enable_jnt_comp)
+ DIFF(enable_ref_frame_mvs)
+ DIFF(seq_force_screen_content_tools)
+ DIFF(seq_force_integer_mv)
+ DIFF(order_hint_bits_minus_1)
+
+ DIFF(enable_superres)
+ DIFF(enable_cdef)
+ DIFF(enable_restoration)
+ DIFF(color_config.high_bitdepth)
+ DIFF(color_config.twelve_bit)
+ DIFF(color_config.mono_chrome)
+ DIFF(color_config.color_description_present_flag)
+ DIFF(color_config.color_primaries)
+ DIFF(color_config.transfer_characteristics)
+ DIFF(color_config.matrix_coefficients)
+ DIFF(color_config.color_range)
+ DIFF(color_config.subsampling_x)
+ DIFF(color_config.subsampling_y)
+ DIFF(color_config.chroma_sample_position)
+ DIFF(color_config.separate_uv_delta_q)
+ DIFF(color_config.i_chroma)
+ DIFF(film_grain_params_present)
+ false
+ )
+ return false;
+
+ for (size_t i=0; i<ARRAY_SIZE(seq1->operating_points); i++)
+ {
+ if (
+ DIFF(operating_points[i].operating_point_idc)
+ DIFF(operating_points[i].seq_level_idx)
+ DIFF(operating_points[i].seq_tier)
+ DIFF(operating_points[i].decoder_model_present_for_this_op)
+ DIFF(operating_points[i].operating_parameters_info.decoder_buffer_delay)
+ DIFF(operating_points[i].operating_parameters_info.encoder_buffer_delay)
+ DIFF(operating_points[i].operating_parameters_info.low_delay_mode_flag)
+ DIFF(operating_points[i].initial_display_delay_present_for_this_op)
+ DIFF(operating_points[i].initial_display_delay_minus_1)
+ false
+ )
+ return false;
+ }
+
+ return true;
+}
=====================================
modules/packetizer/av1_obu.h
=====================================
@@ -167,6 +167,9 @@ bool AV1_get_colorimetry( const av1_OBU_sequence_header_t *,
video_color_primaries_t *, video_transfer_func_t *,
video_color_space_t *, bool *);
bool AV1_get_frame_rate(const av1_OBU_sequence_header_t *, unsigned *, unsigned *);
+vlc_fourcc_t AV1_get_chroma(const av1_OBU_sequence_header_t *);
+
+bool AV1_sequence_header_equal(const av1_OBU_sequence_header_t *,const av1_OBU_sequence_header_t *);
View it on GitLab: https://code.videolan.org/videolan/vlc/-/compare/9c1c1edb451403e81650dfd249a3084c1acb9c64...99b71d93a0f8b322caa04cc8ef2b4c2be60ea1e5
--
View it on GitLab: https://code.videolan.org/videolan/vlc/-/compare/9c1c1edb451403e81650dfd249a3084c1acb9c64...99b71d93a0f8b322caa04cc8ef2b4c2be60ea1e5
You're receiving this email because of your account on code.videolan.org.
VideoLAN code repository instance
More information about the vlc-commits
mailing list