[vlc-devel] [PATCH] libvlc: the setup/cleanup rendering callbacks can be called many times

[Steve Lhomme]

In some cases the decoder might ask for a decoder device that will use the
setup callback. But in the end it's not using the decoder device. Then the
display module is created, without knowing about that decoder device (via a
video context) and calls the setup callback again.

Unless the decoder device that is created in the vout is passed to the display
module, we need to allow those multiple calls per session.

Fix the D3D11 sample to take this in account. The D3D9 sample already takes
care of it.
---
 doc/libvlc/d3d11_player.cpp       | 10 +++++++---
 include/vlc/libvlc_media_player.h |  3 +++
 2 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/doc/libvlc/d3d11_player.cpp b/doc/libvlc/d3d11_player.cpp
index 2cac37a36e0..4cfdce4d51b 100644
--- a/doc/libvlc/d3d11_player.cpp
+++ b/doc/libvlc/d3d11_player.cpp
@@ -475,9 +475,8 @@ static bool Setup_cb( void **opaque, const libvlc_video_setup_device_cfg_t *cfg,
 {
     struct render_context *ctx = static_cast<struct render_context *>(*opaque);
 
-    init_direct3d(ctx);
-
     out->d3d11.device_context = ctx->d3dctxVLC;
+    ctx->d3dctxVLC->AddRef();
     return true;
 }
 
@@ -485,7 +484,7 @@ static void Cleanup_cb( void *opaque )
 {
     // here we can release all things Direct3D11 for good (if playing only one file)
     struct render_context *ctx = static_cast<struct render_context *>( opaque );
-    release_direct3d(ctx);
+    ctx->d3dctxVLC->Release();
 }
 
 static void Resize_cb( void *opaque,
@@ -635,6 +634,8 @@ int WINAPI WinMain(HINSTANCE hInstance,
 
     ShowWindow(Context.hWnd, nCmdShow);
 
+    init_direct3d(&Context);
+
     // DON'T use with callbacks libvlc_media_player_set_hwnd(p_mp, hWnd);
 
     /* Tell VLC to render into our D3D11 environment */
@@ -663,6 +664,9 @@ int WINAPI WinMain(HINSTANCE hInstance,
 
     libvlc_media_player_release( Context.p_mp );
     libvlc_media_release( p_media );
+
+    release_direct3d(&Context);
+
     libvlc_release( p_libvlc );
 
     DeleteCriticalSection(&Context.sizeLock);
diff --git a/include/vlc/libvlc_media_player.h b/include/vlc/libvlc_media_player.h
index 77cebe6d57c..1544b5cc302 100644
--- a/include/vlc/libvlc_media_player.h
+++ b/include/vlc/libvlc_media_player.h
@@ -748,6 +748,9 @@ typedef bool( *libvlc_video_output_select_plane_cb )( void *opaque, size_t plane
  * \param select_plane_cb callback to select different D3D11 rendering targets
  * \param opaque private pointer passed to callbacks
  *
+ * \note the \param setup_cb and \param cleanup_cb may be called more than once per
+ * playback.
+ *
  * \retval true engine selected and callbacks set
  * \retval false engine type unknown, callbacks not set
  * \version LibVLC 4.0.0 or later
-- 
2.29.2