[vlc-devel] [PATCH v3 00/10] Allow the vout_window implementation to suspend rendering

[Alexandre Janniaux]

This patchset enables the vout_window implementation to signal
the visibility state to the vout thread, which can then suspend
the rendering with the vout_display. This is primarily designed
for iOS where the application can be moved to the background and
should not submit any draw commands from there, but it should
also match other cases where the application can stop doing the
rendering, for instance because the application is not visible
anymore, and thus should also be usable in cases like Wayland as
long as the correct feedback is implemented.

It will also suspend rendering as soon as the window is resized
to a zero size, which can happen on iOS since UIView size is
made of floating values and a size between 0 and 1 is a valid
size. However, it's of course unsupported by OpenGL which cannot
create a zero-sized framebuffer and was usually raising assertion
in that case. Other window systems tend to also treat 0x0 as a
special case to avoid, leading to fatal (crash) protocol errors
with wayland subsurfaces for instance. It could be handled by the
display but since the case seems common, let it be handled by the
vout thread directly.

The main goal of this patchset is to move the responsability of
the rendering suspension from the vlc_gl_t implementation in
VLCOpenGLES2VideoView.m to the vout_window implementation which
is usually the first receiver of such information. In addition,
the vlc_gl_t cannot really suspend the rendering other than
making the vlc_gl_MakeCurrent call fail, which is badly handled
by the client of vlc_gl_t and sometimes even impossible like
opengl/display.c:Close. As mentioned a while ago on this mailing
list, iirc by Rémi, we should prevent this function from failing
especially since the reasons of failure are mostly bad
programming situations. This eliminates the failures in the call
to MakeCurrent, and potentially defers it to the call to the
function vlc_gl_Resize() -- which currently doesn't handle
failure but should handle it so that display actually signals
that it cannot resize to the size given to the DISPLAY_SIZE
change control.

To be precise, this patchset fixes two assertions in these cases,
happening when the iOS/tvOS application is starting/closing the
opengl display from the background, or resized to invalid sizes.
The first happens in the VLCOpenGLES2VideoView module because
vlc_gl_ReleaseCurrent is called even if vlc_gl_MakeCurrent has
failed. The second happens in subsequent opengl rendering code in
debug because the call to [VLCOpenGLES2VideoView doResetBuffers]
leads to incomplete framebuffers, stacking the error in the OpenGL
context, and the error is then triggering GL_ASSERT_NO_ERROR in the
OpenGL rendering code.

Changes since v2:

 - renamed suspension/resumed callbacks to visibilityChanged.

 - added a picture_Released when vout_display_Display() is not
   called, fixing the picture leak.

 - added resize test in iosvlc.m.

 - added the handling of the 0x_ / _x0 sizes for rendering.

 - moved resize handling in VLCOpenGLES2VideoView from MakeCurrent
   to Resize().

 - changed the display_lock to render_lock for enabling rendering,
   to avoid blocking the display creation on iOS.

Alexandre Janniaux (10):
  test: iosvlc: use subview for rendering
  test: iosvlc: add pinch gesture recognizer
  vout_window: add visibility reporting callbacks
  video_output: conditionnally enable display rendering
  video_output: window: forward rendering suspending/resuming callbacks
  video_output: conditionnally render on display
  video_output: disable rendering on empty size
  VLCVideoUIView: handle application backgrounding
  VLCOpenGLES2VideoView: handle state change
  VLCOpenGLES2VideoView: handle UIView resize

 include/vlc_vout_window.h                     | 33 +++++++
 .../apple/VLCOpenGLES2VideoView.m             | 96 ++++++++++---------
 modules/video_output/apple/VLCVideoUIView.m   | 28 ++++++
 src/video_output/video_output.c               | 38 +++++++-
 src/video_output/vout_internal.h              |  1 +
 src/video_output/window.c                     | 21 ++++
 test/iosvlc.m                                 | 25 ++++-
 7 files changed, 189 insertions(+), 53 deletions(-)

--
2.30.1