[vlc-commits] [Git][videolan/vlc][master] 13 commits: transcode: unload the decoder before cleaning fmt_in

Steve Lhomme (@robUx4) gitlab at videolan.org
Fri Aug 21 06:07:33 UTC 2026



Steve Lhomme pushed to branch master at VideoLAN / VLC


Commits:
2b807d98 by Alexandre Janniaux at 2026-08-21T05:52:57+00:00
transcode: unload the decoder before cleaning fmt_in

When cleaning p_owner->fmt_in format, the dec->fmt_in pointer points to
memory that becomes reset. But since the decoder is still alive, it can
be running a thread that checks fmt_in between the format clean and the
module unloading, leading to unexpected values and then, crashes in
decoder_QueueVideo.

Every calls were refactored in bfa9a41cac92385ee139aec47592f6541b864c13
but the order was wrong even before. On the input side, it was already
fixed with commit 3007bc4a7b37654c35e087af94b5cd68381fbf27 and commit
2e928f40f1c8fd6656a9527f623cd3bdb1647f33.

Regression from 08b24866dc443c30df8f9d5ee4f7e2045bf62956 which
introduced the cleaning before the module is unloaded.

Refs #27532

- - - - -
2a04abfc by Alexandre Janniaux at 2026-08-21T05:52:57+00:00
test: transcode: check that fmt_in outlives the decoder

A decoder module can read the format it was opened with for as long as
it is loaded, and a decoder that produces frames from its own thread can
still be queueing pictures right when the decoder is being asked to
stop.

We can check in the close callback that the condition holds, since
fmt_in is supposed to never be cleaned up before the close callback is
called.

Refs #27532

- - - - -
22e6ede6 by Alexandre Janniaux at 2026-08-21T05:52:57+00:00
image: unload the decoder before cleaning fmt_in

When using decoder_Init(), dec->fmt_in reference the format owned by the
decoder owner, so the decoder must be unloaded before this format is
cleaned up since the decoder can access it at any point.

The image handler was cleaning that format first, and only then calling
decoder_Destroy(), which unloads the module. The order cannot be swapped
while using decoder_Destroy() since it also deletes the object holding
the format, so it is rewritten as decoder_Clean() and
vlc_object_delete().

Same resolution as 3007bc4a7b37654c35e087af94b5cd68381fbf27 and
2e928f40f1c8fd6656a9527f623cd3bdb1647f33.

Refs #27532

- - - - -
fff695b0 by Alexandre Janniaux at 2026-08-21T05:52:57+00:00
mosaic_bridge: unload the decoder before cleaning fmt_in

When using decoder_Init(), dec->fmt_in reference the format owned by the
decoder owner, so the decoder must be unloaded before this format is
cleaned up since the decoder can access it at any point.

ReleaseDecoder() was cleaning both the input and the output formats
before decoder_Clean(), leaving the module reading a reset format until
it gets unloaded. The decoder is not running its own thread here, but
some modules do decode from a thread they own.

Same resolution as 3007bc4a7b37654c35e087af94b5cd68381fbf27 and
2e928f40f1c8fd6656a9527f623cd3bdb1647f33.

Refs #27532

- - - - -
15dc692f by Alexandre Janniaux at 2026-08-21T05:52:57+00:00
vpx_alpha: unload the sub-decoders before cleaning fmt_in

When using decoder_Init(), dec->fmt_in reference the format owned by the
decoder owner, so the decoder must be unloaded before this format is
cleaned up since the decoder can access it at any point.

CloseDecoder() was cleaning the formats of both sub-decoders before
destroying them. The formats cannot be cleaned after decoder_Destroy(),
which also deletes the object storing them, so rewrite it as
decoder_Clean() and vlc_object_delete().

Same resolution as 3007bc4a7b37654c35e087af94b5cd68381fbf27 and
2e928f40f1c8fd6656a9527f623cd3bdb1647f33.

Refs #27532

- - - - -
c4f869c2 by Alexandre Janniaux at 2026-08-21T05:52:57+00:00
vpx_alpha: clean the sub-decoder formats when opening fails

The error paths of OpenDecoder() were only destroying the sub-decoders,
leaking the input format that decoder_Init() copied into the owner as
well as the output format.

Reuse the destruction sequence from CloseDecoder() so both paths clean
the same fields, in the order where the module is unloaded before the
format it can read is cleaned.

- - - - -
7d126ce3 by Alexandre Janniaux at 2026-08-21T05:52:57+00:00
test: decoder: unload the decoder before cleaning fmt_in

decoder_Init() makes dec->fmt_in reference the format owned by the decoder
owner, so that storage must outlive the module: a loaded module can read
its input format at any point.

The decoder test helper was cleaning that format first everywhere it
tears a decoder down, including the reload path where a module is
actually loaded. The format cannot be cleaned after decoder_Destroy(),
which also deletes the object storing it, so open-code it as
decoder_Clean() and vlc_object_delete().

Same resolution as 3007bc4a7b37654c35e087af94b5cd68381fbf27 and
2e928f40f1c8fd6656a9527f623cd3bdb1647f33.

Refs #27532

- - - - -
2febc748 by Alexandre Janniaux at 2026-08-21T05:52:57+00:00
input: decoder: unload the packetizer before cleaning its fmt_in

When using decoder_Init(), dec->fmt_in reference the format owned by the
decoder owner, so the decoder must be unloaded before this format is
cleaned up since the decoder can access it at any point. Packetizers
don't have this issue since they are always synchronous, but they share
the decoder_t structure and we'll be removing the decoder_Destroy()
helper function, so let's adapt the same way and ensure the pattern stays
correct everywhere.

Same resolution as 3007bc4a7b37654c35e087af94b5cd68381fbf27 and
2e928f40f1c8fd6656a9527f623cd3bdb1647f33.

Refs #27532

- - - - -
103fea38 by Alexandre Janniaux at 2026-08-21T05:52:57+00:00
sdi: release the decoder device before the decoder owner

AbstractDecodedStream::ReleaseDecoder() destroys the decoder and resets
p_decoder to NULL. VideoDecodedStream::ReleaseDecoder() was calling it
first, and then using container_of() on the NULL p_decoder to reach the
decoder device stored in the owner, dereferencing an invalid pointer on
every video stream teardown.

Release the device while the owner is still valid. Since the device is
reference counted, the decoder will still be able to use it.

The override was introduced in 81512e0382f289de5104e213373e044095d73f8c,
after 6e6bd2608fbb4faff91b273d5ea2054ac9e23275 moved the destruction into
ReleaseDecoder(), but it's more natural to have the parent destructor
run after the derived destructor, contrary to constructors.

- - - - -
a33ca2bf by Alexandre Janniaux at 2026-08-21T05:52:57+00:00
sdi: unload the decoder before cleaning fmt_in

decoder_Init() makes dec->fmt_in reference the format owned by the decoder
owner, so that storage must outlive the module: a loaded module can read
its input format at any point, and the SDI output decodes from its own
thread.

ReleaseDecoder() was cleaning the formats before destroying the decoder,
so unload the module first. The formats cannot be cleaned after
decoder_Destroy(), which also deletes the object storing them, so
open-code it as decoder_Clean() and vlc_object_delete().

The thread creation failure path was a copy of ReleaseDecoder() with the
same inversion, so call ReleaseDecoder() there instead.

Same resolution as 3007bc4a7b37654c35e087af94b5cd68381fbf27 and
2e928f40f1c8fd6656a9527f623cd3bdb1647f33.

Refs #27532

- - - - -
0f9b1db6 by Alexandre Janniaux at 2026-08-21T05:52:57+00:00
test: sdi: add non-regression test for decoder

There was no way to test SDI module because of the decklink SDK
availability but decoders spawning was broken for the same reasons
fixed in the commit 3007bc4a7b37654c35e087af94b5cd68381fbf27. However,
the decoder abstractions in the SDI modules are not actually depending
on the SDK itself, so we can actually write a small smoke test for it
that also check the non-regression on the decoder fmt_in handling, as
well as checking how the decoder device is actually provided to the SDI
module.

The test mocks every modules being used but is still written in test/
since it spawns a libvlc instance.

- - - - -
853d975d by Alexandre Janniaux at 2026-08-21T05:52:57+00:00
input: decoder: clean the module before fmt_in on failure

decoder_Init() makes dec->fmt_in reference the format owned by the
decoder owner, so that storage must outlive the module.

This is the last call site from 08b24866dc443c30df8f9d5ee4f7e2045bf62956
still cleaning the format first. At this location, this cannot crash
since the module could not be loaded and decoder_Clean() has nothing to
unload, but let's just fix it so that it's not copy-pasted somewhere
else in the wrong order.

Same resolution as 3007bc4a7b37654c35e087af94b5cd68381fbf27 and
2e928f40f1c8fd6656a9527f623cd3bdb1647f33.

Refs #27532

- - - - -
659b9f12 by Alexandre Janniaux at 2026-08-21T05:52:57+00:00
codec: remove decoder_Destroy()

decoder_Destroy() unloads the module and deletes the decoder object in
one call, but decoder_Init() stores the input format in the owner, which
is usually allocated together with the decoder. Owners then had no way
to clean that format between the two steps. Doing it before left the
module reading a corrupted/reset format, and doing it after was a
use-after-free.

Every caller now uses decoder_Clean(), the format cleanup and
vlc_object_delete(), so drop the helper and document the constraint on
decoder_Clean() instead.

Refs #27532

- - - - -


14 changed files:

- include/vlc_codec.h
- modules/codec/vpx_alpha.c
- modules/stream_out/mosaic_bridge.c
- modules/stream_out/sdi/SDIStream.cpp
- modules/stream_out/transcode/transcode.h
- src/input/decoder.c
- src/input/decoder_helpers.c
- src/libvlccore.sym
- src/misc/image.c
- test/Makefile.am
- test/modules/meson.build
- + test/modules/stream_out/sdi.cpp
- test/modules/stream_out/transcode.c
- test/src/input/decoder.c


Changes:

=====================================
include/vlc_codec.h
=====================================
@@ -413,17 +413,15 @@ VLC_API void decoder_Init( decoder_t *dec, es_format_t *fmt_in, const es_format_
 VLC_API int decoder_LoadModule(decoder_t *dec, bool packetizer,
                                 bool use_varoption);
 
-/**
- * Destroy a decoder and reset the structure.
- *
- * To be used by decoder owners.
- */
-VLC_API void decoder_Destroy( decoder_t *p_dec );
-
 /**
  * Unload a decoder module and reset the input/output formats.
  *
  * To be used by decoder owners.
+ *
+ * \note
+ * The module can read the format given to decoder_Init() until it is
+ * unloaded, so the owner must call this function before cleaning that
+ * format, and delete the decoder object afterwards.
  */
 VLC_API void decoder_Clean( decoder_t *p_dec );
 


=====================================
modules/codec/vpx_alpha.c
=====================================
@@ -565,6 +565,14 @@ static void Flush( decoder_t *dec )
     vlc_mutex_unlock(&p_sys->lock);
 }
 
+static void DestroyDecoder(struct vp_decoder *vpdec)
+{
+    decoder_Clean(&vpdec->dec);
+    es_format_Clean(&vpdec->fmt_out);
+    es_format_Clean(&vpdec->fmt_in);
+    vlc_object_delete(&vpdec->dec);
+}
+
 int OpenDecoder(vlc_object_t *o)
 {
     decoder_t *dec = container_of(o, decoder_t, obj);
@@ -628,16 +636,16 @@ int OpenDecoder(vlc_object_t *o)
     decoder_LoadModule(&p_sys->opaque->dec, false, true);
     if (p_sys->opaque->dec.p_module == NULL)
     {
-        decoder_Destroy(&p_sys->alpha->dec);
-        decoder_Destroy(&p_sys->opaque->dec);
+        DestroyDecoder(p_sys->alpha);
+        DestroyDecoder(p_sys->opaque);
         return VLC_EGENERIC;
     }
     p_sys->alpha->dec.cbs = &dec_cbs;
     decoder_LoadModule(&p_sys->alpha->dec, false, true);
     if (p_sys->alpha->dec.p_module == NULL)
     {
-        decoder_Destroy(&p_sys->alpha->dec);
-        decoder_Destroy(&p_sys->opaque->dec);
+        DestroyDecoder(p_sys->alpha);
+        DestroyDecoder(p_sys->opaque);
         return VLC_EGENERIC;
     }
 
@@ -652,12 +660,8 @@ void CloseDecoder(vlc_object_t *o)
     decoder_t *dec = container_of(o, decoder_t, obj);
     vpx_alpha *p_sys = dec->p_sys;
 
-    es_format_Clean(&p_sys->opaque->fmt_out);
-    es_format_Clean(&p_sys->opaque->fmt_in);
-    decoder_Destroy(&p_sys->opaque->dec);
-    es_format_Clean(&p_sys->alpha->fmt_out);
-    es_format_Clean(&p_sys->alpha->fmt_in);
-    decoder_Destroy(&p_sys->alpha->dec);
+    DestroyDecoder(p_sys->opaque);
+    DestroyDecoder(p_sys->alpha);
 
     if (p_sys->pool)
         picture_pool_Release(p_sys->pool);


=====================================
modules/stream_out/mosaic_bridge.c
=====================================
@@ -142,9 +142,9 @@ static void ReleaseDecoder( decoder_t *p_dec )
         vlc_decoder_device_Release( p_owner->dec_dev );
         p_owner->dec_dev = NULL;
     }
+    decoder_Clean( p_dec );
     es_format_Clean( &p_owner->fmt_in );
     es_format_Clean( &p_owner->fmt_out );
-    decoder_Clean( p_dec );
     if ( p_owner->filters != NULL )
         filter_chain_Delete( p_owner->filters );
     if ( p_owner->vctx != NULL )


=====================================
modules/stream_out/sdi/SDIStream.cpp
=====================================
@@ -280,11 +280,7 @@ bool AbstractDecodedStream::init(const es_format_t *p_fmt)
 
     if(vlc_clone(&thread, decoderThreadCallback, this))
     {
-        es_format_Clean(&p_owner->decoder_out);
-        es_format_Clean(&p_owner->last_fmt_update);
-        es_format_Clean(&p_owner->fmt_in);
-        decoder_Destroy( p_decoder );
-        p_decoder = NULL;
+        ReleaseDecoder();
         return false;
     }
 
@@ -410,10 +406,11 @@ void AbstractDecodedStream::ReleaseDecoder()
 {
     struct decoder_owner *p_owner;
     p_owner = container_of(p_decoder, struct decoder_owner, dec);
+    decoder_Clean(p_decoder);
     es_format_Clean(&p_owner->decoder_out);
     es_format_Clean(&p_owner->last_fmt_update);
     es_format_Clean(&p_owner->fmt_in);
-    decoder_Destroy( p_decoder );
+    vlc_object_delete(p_decoder);
     p_decoder = NULL;
 }
 
@@ -487,8 +484,6 @@ vlc_decoder_device * VideoDecodedStream::VideoDecCallback_get_device(decoder_t *
 
 void VideoDecodedStream::ReleaseDecoder()
 {
-    AbstractDecodedStream::ReleaseDecoder();
-
     struct decoder_owner *p_owner;
     p_owner = container_of(p_decoder, struct decoder_owner, dec);
     if (p_owner->dec_dev)
@@ -496,6 +491,8 @@ void VideoDecodedStream::ReleaseDecoder()
         vlc_decoder_device_Release(p_owner->dec_dev);
         p_owner->dec_dev = NULL;
     }
+
+    AbstractDecodedStream::ReleaseDecoder();
 }
 
 int VideoDecodedStream::VideoDecCallback_update_format(decoder_t *p_dec,


=====================================
modules/stream_out/transcode/transcode.h
=====================================
@@ -179,8 +179,10 @@ static inline void dec_Delete( decoder_t *p_dec )
         return;
 
     struct decoder_owner *p_owner = dec_get_owner( p_dec );
+
+    decoder_Clean(p_dec);
     es_format_Clean( &p_owner->fmt_in );
-    decoder_Destroy( p_dec );
+    vlc_object_delete(p_dec);
 }
 
 static inline void es_format_SetMeta( es_format_t *p_dst, const es_format_t *p_src )


=====================================
src/input/decoder.c
=====================================
@@ -528,8 +528,8 @@ static int LoadDecoder(decoder_t *p_dec, bool b_packetizer, es_format_t *fmt_in)
     decoder_LoadModule( p_dec, b_packetizer, true );
     if( !p_dec->p_module )
     {
-        es_format_Clean(fmt_in);
         decoder_Clean( p_dec );
+        es_format_Clean(fmt_in);
         return -1;
     }
     return 0;
@@ -2322,6 +2322,8 @@ static void DeleteDecoder( vlc_input_decoder_t *p_owner, enum es_format_category
              (char*)&p_dec->fmt_in->i_codec );
 
     decoder_Clean( p_dec );
+    if( p_owner->p_packetizer != NULL )
+        decoder_Clean( p_owner->p_packetizer );
 
     /* Free all packets still in the decoder fifo. */
     block_FifoEmpty( p_owner->p_fifo );
@@ -2398,8 +2400,9 @@ static void DeleteDecoder( vlc_input_decoder_t *p_owner, enum es_format_category
         vlc_meta_Delete( p_owner->p_description );
 
     block_FifoRelease( p_owner->p_fifo );
-    decoder_Destroy( p_owner->p_packetizer );
-    decoder_Destroy( &p_owner->dec );
+    if( p_owner->p_packetizer != NULL )
+        vlc_object_delete( p_owner->p_packetizer );
+    vlc_object_delete( &p_owner->dec );
 }
 
 /* */


=====================================
src/input/decoder_helpers.c
=====================================
@@ -107,15 +107,6 @@ void decoder_Clean( decoder_t *p_dec )
     }
 }
 
-void decoder_Destroy( decoder_t *p_dec )
-{
-    if (p_dec != NULL)
-    {
-        decoder_Clean( p_dec );
-        vlc_object_delete(p_dec);
-    }
-}
-
 int decoder_UpdateVideoFormat( decoder_t *dec )
 {
     return decoder_UpdateVideoOutput( dec, NULL );


=====================================
src/libvlccore.sym
=====================================
@@ -95,7 +95,6 @@ decoder_NewPicture
 decoder_Init
 decoder_LoadModule
 decoder_Clean
-decoder_Destroy
 decoder_NewAudioBuffer
 decoder_UpdateVideoFormat
 decoder_UpdateVideoOutput


=====================================
src/misc/image.c
=====================================
@@ -123,8 +123,9 @@ void image_HandlerDelete( image_handler_t *p_image )
     if( p_image->p_dec != NULL )
     {
         struct decoder_owner *p_owner = dec_get_owner( p_image->p_dec );
+        decoder_Clean( p_image->p_dec );
         es_format_Clean( &p_owner->fmt_in );
-        decoder_Destroy( p_image->p_dec );
+        vlc_object_delete( p_image->p_dec );
     }
     if( p_image->p_enc )
         vlc_encoder_Destroy( p_image->p_enc );
@@ -166,8 +167,9 @@ static picture_t *ImageRead( image_handler_t *p_image, block_t *p_block,
         p_image->p_dec->fmt_in->i_codec != p_es_in->video.i_chroma )
     {
         struct decoder_owner *p_owner = dec_get_owner( p_image->p_dec );
+        decoder_Clean( p_image->p_dec );
         es_format_Clean( &p_owner->fmt_in );
-        decoder_Destroy( p_image->p_dec );
+        vlc_object_delete( p_image->p_dec );
         p_image->p_dec = NULL;
     }
 
@@ -183,8 +185,9 @@ static picture_t *ImageRead( image_handler_t *p_image, block_t *p_block,
         if( p_image->p_dec->fmt_out.i_cat != VIDEO_ES )
         {
             struct decoder_owner *p_owner = dec_get_owner( p_image->p_dec );
+            decoder_Clean( p_image->p_dec );
             es_format_Clean( &p_owner->fmt_in );
-            decoder_Destroy( p_image->p_dec );
+            vlc_object_delete( p_image->p_dec );
             p_image->p_dec = NULL;
             block_Release(p_block);
             return NULL;
@@ -692,8 +695,9 @@ static decoder_t *CreateDecoder( image_handler_t *p_image, const es_format_t *fm
                  "VLC probably does not support this image format.",
                  (char*)&p_dec->fmt_in->i_codec );
 
+        decoder_Clean( p_dec );
         es_format_Clean( &p_owner->fmt_in );
-        decoder_Destroy( p_dec );
+        vlc_object_delete( p_dec );
         p_dec = NULL;
     }
 


=====================================
test/Makefile.am
=====================================
@@ -91,6 +91,7 @@ check_PROGRAMS = \
 	test_modules_demux_ts_pes \
 	test_modules_playlist_m3u \
 	test_modules_stream_out_pcr_sync \
+	test_modules_stream_out_sdi \
 	test_modules_tls \
 	test_modules_stream_out_transcode \
 	test_modules_mux_webvtt \
@@ -450,6 +451,11 @@ test_modules_stream_out_pcr_sync_SOURCES = modules/stream_out/pcr_sync.c \
 	../modules/stream_out/transcode/pcr_helper.c
 test_modules_stream_out_pcr_sync_LDADD = $(LIBVLCCORE)
 
+test_modules_stream_out_sdi_SOURCES = modules/stream_out/sdi.cpp \
+	../modules/stream_out/sdi/SDIStream.cpp \
+	../modules/stream_out/sdi/SDIStream.hpp
+test_modules_stream_out_sdi_LDADD = $(LIBVLCCORE) $(LIBVLC)
+
 test_modules_mux_webvtt_SOURCES = modules/mux/webvtt.c
 test_modules_mux_webvtt_LDADD = $(LIBVLCCORE) $(LIBVLC)
 


=====================================
test/modules/meson.build
=====================================
@@ -228,6 +228,16 @@ vlc_tests += {
     'module_depends' : vlc_plugins_targets.keys()
 }
 
+vlc_tests += {
+    'name' : 'test_modules_stream_out_sdi',
+    'sources' : files(
+        'stream_out/sdi.cpp',
+        '../../modules/stream_out/sdi/SDIStream.cpp',
+        '../../modules/stream_out/sdi/SDIStream.hpp'),
+    'suite' : ['modules', 'test_modules'],
+    'link_with' : [libvlc, libvlccore]
+}
+
 vlc_tests += {
     'name' : 'test_modules_stream_out_pcr_sync',
     'sources' : files(


=====================================
test/modules/stream_out/sdi.cpp
=====================================
@@ -0,0 +1,220 @@
+/*****************************************************************************
+ * sdi.cpp: test for the SDI output decoded streams
+ *****************************************************************************
+ * Copyright (C) 2026 VideoLabs
+ *
+ * Author: Alexandre Janniaux <ajanni at videolabs.io>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; either version 2.1 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin Street, Fifth Floor, Boston MA 02110-1301, USA.
+ *****************************************************************************/
+
+#ifdef HAVE_CONFIG_H
+# include "config.h"
+#endif
+
+/* Define a builtin module for mocked parts */
+#define MODULE_NAME test_sdi_mock
+#undef VLC_DYNAMIC_PLUGIN
+
+#include "../../libvlc/test.h"
+#include <vlc_common.h>
+#include <vlc_plugin.h>
+#include <vlc_codec.h>
+
+#include "../../../modules/stream_out/sdi/SDIStream.hpp"
+#include "../lib/libvlc_internal.h"
+
+#include <climits>
+#include <cstdint>
+#include <memory>
+
+const char vlc_module_name[] = MODULE_STRING;
+
+static unsigned decoder_opened = 0;
+static unsigned decoder_closed = 0;
+
+struct dec_fmt {
+    vlc_fourcc_t codec;
+};
+
+static void DecoderDeviceClose(struct vlc_decoder_device *device)
+    { VLC_UNUSED(device); }
+
+static const struct vlc_decoder_device_operations decoder_device_ops =
+{
+    DecoderDeviceClose,
+};
+
+static int OpenDecoderDevice(struct vlc_decoder_device *device,
+                             vlc_window_t *window)
+{
+    VLC_UNUSED(window);
+    device->ops = &decoder_device_ops;
+    /* The private parts are never used in this test. */
+    device->type = VLC_DECODER_DEVICE_VAAPI;
+    return VLC_SUCCESS;
+}
+
+static int DecoderDecode(decoder_t *dec, vlc_frame_t *frame)
+{
+    VLC_UNUSED(dec);
+    if (frame != NULL)
+        vlc_frame_Release(frame);
+    return VLCDEC_SUCCESS;
+}
+
+static void CloseDecoder(vlc_object_t *obj)
+{
+    decoder_t *dec = reinterpret_cast<decoder_t *>(obj);
+    dec_fmt *sys = static_cast<dec_fmt*>(dec->p_sys);
+
+    /* An owner cleaning fmt_in before unloading the module would leave an
+     * UNKNOWN_ES format behind. */
+    assert(dec->fmt_in->i_cat == VIDEO_ES || dec->fmt_in->i_cat == AUDIO_ES);
+    assert(dec->fmt_in->i_codec == sys->codec);
+
+    decoder_closed++;
+    delete sys;
+}
+
+static int OpenDecoder(vlc_object_t *obj)
+{
+    decoder_t *dec = reinterpret_cast<decoder_t *>(obj);
+
+    if (dec->fmt_in->i_cat == VIDEO_ES)
+    {
+        /* Make the owner instantiate its decoder device, so that the
+         * teardown has to release it from the owner storage. */
+        vlc_decoder_device *dev = decoder_GetDecoderDevice(dec);
+        assert(dev != NULL);
+        vlc_decoder_device_Release(dev);
+    }
+
+    es_format_Clean(&dec->fmt_out);
+    es_format_Copy(&dec->fmt_out, dec->fmt_in);
+
+    /* Remember the format the module was opened with, so that the close
+     * callback can check it is still the one referenced by fmt_in. */
+    dec_fmt *sys = new dec_fmt;
+    sys->codec = dec->fmt_in->i_codec;
+    dec->p_sys = reinterpret_cast<void *>(sys);
+    dec->pf_decode = DecoderDecode;
+
+    decoder_opened++;
+    return VLC_SUCCESS;
+}
+
+static void test_decoded_stream(vlc_object_t *obj, vlc_fourcc_t i_codec,
+                                es_format_category_e i_cat)
+{
+    std::unique_ptr<sdi_sout::AbstractStreamOutputBuffer> buffer;
+    std::unique_ptr<sdi_sout::AbstractDecodedStream> stream;
+    es_format_t output;
+
+    const sdi_sout::StreamID id(1);
+
+    if (i_cat == VIDEO_ES)
+    {
+        buffer = std::make_unique<sdi_sout::PictureStreamOutputBuffer>();
+        stream = std::make_unique<sdi_sout::VideoDecodedStream>(obj, id, buffer.get());
+
+        es_format_Init(&output, VIDEO_ES, VLC_CODEC_UYVY);
+        output.video.i_chroma = output.i_codec;
+        output.video.i_width = output.video.i_visible_width = 720;
+        output.video.i_height = output.video.i_visible_height = 576;
+    }
+    else
+    {
+        buffer = std::make_unique<sdi_sout::BlockStreamOutputBuffer>();
+        stream = std::make_unique<sdi_sout::AudioDecodedStream>(obj, id, buffer.get());
+
+        es_format_Init(&output, AUDIO_ES, VLC_CODEC_S16N);
+        output.audio.i_format = output.i_codec;
+        output.audio.i_rate = 48000;
+        output.audio.i_physical_channels = AOUT_CHANS_STEREO;
+        aout_FormatPrepare(&output.audio);
+    }
+
+    stream->setOutputFormat(&output);
+    es_format_Clean(&output);
+
+    es_format_t input;
+    es_format_Init(&input, i_cat, i_codec);
+    if (i_cat == VIDEO_ES)
+    {
+        input.video.i_width = input.video.i_visible_width = 720;
+        input.video.i_height = input.video.i_visible_height = 576;
+    }
+    else
+    {
+        input.audio.i_rate = 48000;
+        input.audio.i_physical_channels = AOUT_CHANS_STEREO;
+    }
+
+    const unsigned opened = decoder_opened;
+    const unsigned closed = decoder_closed;
+
+    bool success = stream->init(&input);
+    assert(success);
+    assert(decoder_opened == opened + 1);
+
+    /* Check decoder unloading. */
+    stream.reset();
+    assert(decoder_closed == closed + 1);
+
+    es_format_Clean(&input);
+    buffer.reset();
+}
+
+vlc_module_begin()
+    set_callbacks(OpenDecoder, CloseDecoder)
+    set_capability("video decoder", INT_MAX)
+
+    add_submodule()
+        set_callbacks(OpenDecoder, CloseDecoder)
+        set_capability("audio decoder", INT_MAX)
+
+    add_submodule()
+        set_callback_dec_device(OpenDecoderDevice, 0)
+vlc_module_end()
+
+extern "C" VLC_EXPORT const vlc_plugin_cb vlc_static_modules[];
+extern "C" const vlc_plugin_cb vlc_static_modules[] = {
+    VLC_SYMBOL(vlc_entry),
+    NULL
+};
+
+int main(int argc, char **argv)
+{
+    VLC_UNUSED(argc); VLC_UNUSED(argv);
+    test_init();
+
+    const char * const args[] = {
+        "-vv", "--codec=" MODULE_STRING, "--dec-dev=" MODULE_STRING,
+        "--no-auto-preparse",
+    };
+
+    libvlc_instance_t *vlc = libvlc_new(ARRAY_SIZE(args), args);
+    assert(vlc != NULL);
+
+    vlc_object_t *obj = VLC_OBJECT(vlc->p_libvlc_int);
+    test_decoded_stream(obj, VLC_CODEC_MP4V, VIDEO_ES);
+    test_decoded_stream(obj, VLC_CODEC_MPGA, AUDIO_ES);
+
+    libvlc_release(vlc);
+    assert(decoder_opened == 2);
+    assert(decoder_closed == 2);
+    return 0;
+}


=====================================
test/modules/stream_out/transcode.c
=====================================
@@ -103,6 +103,12 @@ static int DecoderDecode(decoder_t *dec, vlc_frame_t *frame)
 static void CloseDecoder(vlc_object_t *obj)
 {
     decoder_t *dec = (decoder_t*)obj;
+
+    /* An owner that cleans fmt_in before unloading would set the category
+     * to UNKNOWN_ES, but it must only be cleaned after this callback. */
+    assert(dec->fmt_in->i_cat == VIDEO_ES);
+    assert(dec->fmt_in->i_codec != 0);
+
     struct vlc_video_context *vctx = dec->p_sys;
     if (vctx)
         vlc_video_context_Release(vctx);


=====================================
test/src/input/decoder.c
=====================================
@@ -107,8 +107,8 @@ static int decoder_load(decoder_t *decoder, bool is_packetizer,
     decoder_LoadModule(decoder, is_packetizer, false);
     if (!decoder->p_module)
     {
-        es_format_Clean( &owner->fmt_in );
         decoder_Clean( decoder );
+        es_format_Clean( &owner->fmt_in );
         return VLC_EGENERIC;
     }
     return VLC_SUCCESS;
@@ -117,8 +117,9 @@ static int decoder_load(decoder_t *decoder, bool is_packetizer,
 static void decoder_destroy_clean(decoder_t *decoder)
 {
     struct decoder_owner *owner = dec_get_owner(decoder);
+    decoder_Clean(decoder);
     es_format_Clean(&owner->fmt_in);
-    decoder_Destroy(decoder);
+    vlc_object_delete(decoder);
 }
 
 void test_decoder_destroy(decoder_t *decoder)
@@ -227,8 +228,8 @@ int test_decoder_process(decoder_t *decoder, block_t *p_block)
             decoder->pf_decode(decoder, NULL);
 
             /* Reload decoder */
-            es_format_Clean( &owner->fmt_in );
             decoder_Clean(decoder);
+            es_format_Clean( &owner->fmt_in );
             if (decoder_load(decoder, false, &packetizer->fmt_out) != VLC_SUCCESS)
             {
                 if (p_block != NULL)



View it on GitLab: https://code.videolan.org/videolan/vlc/-/compare/367f021c2930dca027aa4cd8fdc5d9ebf8d2d541...659b9f122c6f3674749b68ebc58b1a6b34bd79ba

-- 
View it on GitLab: https://code.videolan.org/videolan/vlc/-/compare/367f021c2930dca027aa4cd8fdc5d9ebf8d2d541...659b9f122c6f3674749b68ebc58b1a6b34bd79ba
You're receiving this email because of your account on code.videolan.org. Manage all notifications: https://code.videolan.org/-/profile/notifications | Help: https://code.videolan.org/help




More information about the vlc-commits mailing list