[vlc-commits] glx: force glx usage within the same X11 window instance

[Thomas Guillem]

vlc/vlc-3.0 | branch: master | Thomas Guillem <thomas at gllm.fr> | Tue Mar 13 16:47:13 2018 +0100| [aeb319f7961a04233c6089aa778b9b5b41edec90] | committer: Thomas Guillem

glx: force glx usage within the same X11 window instance

This commit fixes a crash on Debian Stretch and Ubuntu 17.10 when you fallback
from VDPAU (GLX forced) to SW decoding (EGL by default).

Fixes #19982

(cherry picked from commit 95d43f8a3f4904d4f1b895d4ea2092f88b490dc5)
Signed-off-by: Thomas Guillem <thomas at gllm.fr>

> http://git.videolan.org/gitweb.cgi/vlc/vlc-3.0.git/?a=commit;h=aeb319f7961a04233c6089aa778b9b5b41edec90
---

 modules/video_output/glx.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/modules/video_output/glx.c b/modules/video_output/glx.c
index dc692e9a59..cb5d53b7db 100644
--- a/modules/video_output/glx.c
+++ b/modules/video_output/glx.c
@@ -237,6 +237,16 @@ static int Open (vlc_object_t *obj)
     ReleaseCurrent (gl);
 #endif
 
+    /* XXX: Prevent other gl backends (like EGL) to be opened within the same
+     * X11 window instance. Indeed, using EGL after GLX on the same X11 window
+     * instance leads to an SEGFAULT in the libEGL_nvidia.so library. */
+    const char *vendor = glXGetClientString(dpy, GLX_VENDOR);
+    if (vendor && strncmp(vendor, "NVIDIA", sizeof("NVIDIA") - 1) == 0)
+    {
+        var_Create(gl->surface, "gl", VLC_VAR_STRING);
+        var_SetString(gl->surface, "gl", "glx");
+    }
+
     return VLC_SUCCESS;
 
 error: