[vlc-devel] DXVA2 - hw surfaces not being released

[Oliver Collyer]

This is probably one for Steve Lhomme.

The recent changes seem to have introduced a problem in after the DXVA2 hardware decoder has been closed, it hasn't released all the surfaces.

I've discovered this because in my custom app (libvlc-based) it's no longer possible to reset the D3D9 device after playing a video in VLC.

Previously my app used the code from 14th May which was storing the surfaces differently, but didn't have this problem.

If I hack the DxDestroyVideoDecoder function as below, then I can workaround the problem, but clearly it needs a proper fix.

I'm happy to carry on trying to work it out, but you may be able to spot the flaw easier than me :)

static void DxDestroyVideoDecoder(vlc_va_t *va)
{
   directx_sys_t *dx_sys = &va->sys->dx_sys;
   if (dx_sys->decoder)
   {
       IDirectXVideoDecoder_Release(dx_sys->decoder);
       dx_sys->decoder = NULL;
+        for (unsigned i = 0; i < dx_sys->va_pool.surface_count; i++) {
+            unsigned long refs = IDirect3DSurface9_Release(dx_sys->hw_surface[i]);
+            msg_Dbg(va, "DxDestroyVideoDecoder()) - hw_surface[i] released and now has %lu references", refs);
+            while (refs) refs = IDirect3DSurface9_Release(dx_sys->hw_surface[i]);
+            msg_Dbg(va, "DxDestroyVideoDecoder()) - hw_surface[i] released and now has %lu references (forced released)", refs);
+        }
   }
}

The above results in the following trace snippet:

core debug: removing module "avcodec"
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 2 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 1 references
dxva2 debug: DxDestroyVideoDecoder()) - hw_surface[i] released and now has 0 references (forced released)