[vlc-commits] [Git][videolan/vlc][master] qt: disable mask instead of using the exact `QWindow::geometry()` in `CompositorWayland`

[Steve Lhomme (@robUx4)]

Steve Lhomme pushed to branch master at VideoLAN / VLC


Commits:
4ead38ef by Fatih Uzunoglu at 2025-04-19T09:42:31+00:00
qt: disable mask instead of using the exact `QWindow::geometry()` in `CompositorWayland`

This is useful when toggling the window title bar setting. For example, if it is
switched on from off, a title bar appears and the window geometry no longer counts
the title bar, but then, the mask needs to account for the title bar. This is
observed with KWin, where SSD is supported (through `zxdg_decoration_manager_v1`)
but not effective when the setting is switched (which effectively means switching
the `FramelessWindowHint` flag), as in that case Qt platform plugin tries to decorate
the window itself (probably the protocol does not support switching SSD like that).

I assume this occurs because the title bar is not really server-side decoration
but rather the one provided by platform decoration (`QT_WAYLAND_DECORATION`), so
technically client-side decoration (managed by the platform plugin).

Since when `FramelessWindowHint` is off; either SSD (`zxdg_decoration_manager_v1`),
or CSD (managed by the platform plugin), in both cases the `windowExtendedMargin()`
would be set to 0 in our case, this is a valid workaround instead of trying to get
the client side margins with (`QWaylandWindow::clientSideMargins()`) and accounting
it when applying the margins.

I'm not sure if this is a Qt or KWin Wayland bug, because if CSD is handled by the
platform plugin, the application should not be forced to care about the CSD. In that
case for the application, it is no different than SSD.

Also use `QRect(QPoint(0, 0), m_qmlView->size())` instead of `m_qmlView->geometry()`
because mask should be relative to the window, not screen. Currently this does not
change anything because XDG surface (at least regular ones) does not provide where
it is positioned in the screen, so `QWindow::position()` returns (0, 0).

- - - - -


1 changed file:

- modules/gui/qt/maininterface/compositor_wayland.cpp


Changes:

=====================================
modules/gui/qt/maininterface/compositor_wayland.cpp
=====================================
@@ -250,11 +250,24 @@ void CompositorWayland::adjustQuickWindowMask()
 {
     assert(m_intf);
     assert(m_intf->p_mi);
-    unsigned maskMargin = 0;
-    if (Q_LIKELY(static_cast<unsigned>(m_intf->p_mi->CSDBorderSize()) < m_intf->p_mi->windowExtendedMargin()))
-        maskMargin = m_intf->p_mi->windowExtendedMargin() - m_intf->p_mi->CSDBorderSize();
-    const QMargins maskMargins(maskMargin, maskMargin, maskMargin, maskMargin);
-    m_qmlView->setMask(m_qmlView->geometry().marginsRemoved(maskMargins));
+    // Assuming no overflow:
+    const auto maskMargin = static_cast<int>(m_intf->p_mi->windowExtendedMargin()) - m_intf->p_mi->CSDBorderSize();
+    if (maskMargin > 0)
+    {
+        // WARNING: Be careful here, if Qt platform plugin applies decoration (CSD handled by the Qt platform plugin),
+        //          it might be necessary to care about `QWaylandWindow::clientSideMargins()` when setting the mask.
+        //          Since this branch is only taken when FramelessWindowHint (CSD handled by the application) is used,
+        //          we do not need to care about that here.
+        const QMargins maskMargins(maskMargin, maskMargin, maskMargin, maskMargin);
+        m_qmlView->setMask(QRect(QPoint(0, 0), m_qmlView->size()).marginsRemoved(maskMargins));
+    }
+    else
+    {
+        // WARNING: As per the warning above, if you want to set a mask in this case (FramelessWindowHint is off),
+        //          you might need to care about `QWaylandWindow::clientSideMargins()`, unless the mask is disabled
+        //          altogether, as done here:
+        m_qmlView->setMask(QRegion()); // null/empty region, disable mask
+    }
 }
 #endif
 



View it on GitLab: https://code.videolan.org/videolan/vlc/-/commit/4ead38efafafb9afaeb2629a239e0682a0a8abbc

-- 
View it on GitLab: https://code.videolan.org/videolan/vlc/-/commit/4ead38efafafb9afaeb2629a239e0682a0a8abbc
You're receiving this email because of your account on code.videolan.org.


VideoLAN code repository instance