[vlc-commits] mft: fix probe issue with GUI threads

[Erwan Tulou]

vlc/vlc-3.0 | branch: master | Erwan Tulou <erwan10 at videolan.org> | Sat Jan 20 20:49:56 2018 +0100| [8719301547ff6312271458dde1d7b1cc2549ee76] | committer: Erwan Tulou

mft: fix probe issue with GUI threads

The skins2 thread as a Gui interface uses OLE/COM set up as 'appartment threaded' for DragNDrop. This thread also uses the vlc decoding facilities to decode still images and therefore is likely to probe decoder modules.

Calling CoInitializeEx() in multithreaded mode is not compatible with the already 'appartment threaded' setting, and therefore fails. In debug mode, this causes an unreachable assert. In non debug mode, the Open() is executed, fails and calls an undue CoUninitialize() which silently breaks the whole OLE setting of the thread.

The patch just gracefully returns on failure.

(cherry picked from commit 545124d374e5a00c74ce58c40dbc254abc1178cd)
Signed-off-by: Erwan Tulou <erwan10 at videolan.org>

> http://git.videolan.org/gitweb.cgi/vlc/vlc-3.0.git/?a=commit;h=8719301547ff6312271458dde1d7b1cc2549ee76
---

 modules/codec/mft.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/modules/codec/mft.c b/modules/codec/mft.c
index f91fc7c00f..0305300d9c 100644
--- a/modules/codec/mft.c
+++ b/modules/codec/mft.c
@@ -1121,7 +1121,10 @@ static int Open(vlc_object_t *p_this)
         return VLC_ENOMEM;
 
     if( FAILED(CoInitializeEx(NULL, COINIT_MULTITHREADED)) )
-        vlc_assert_unreachable();
+    {
+        free(p_sys);
+        return VLC_EGENERIC;
+    }
 
     if (LoadMFTLibrary(&p_sys->mf_handle))
     {