[vlc-devel] [PATCH 4/4] [3.x] vout/caopengllayer: rewrite most of the module

Thomas Guillem thomas at gllm.fr
Thu Feb 20 13:57:00 CET 2020


Hello,

I have few remarks onlined:

On Thu, Feb 20, 2020, at 11:27, Marvin Scholz wrote:
> Rewrites most of the layer vout code to have the same features as the
> view based vout. Additionally fixes laggy resizing, fixes CGL context
> creation bugs, adds support for CI filters and fixes various memory
> management errors.
> 
> The CAOpenGLLayer based API is special and different from all other APIs
> provided on other OSes as it is not a push-model API but a pull one,
> where the OS calls a specific method when a new frame should be rendered.
> This makes integration into VLC relatively tricky and the code a bit
> harder to follow.
> While the API is a pull-model, we can kind of trick it by just forcing
> a re-display of the layer in the vouts display function. With views this
> would be forbidden as views are supposed to be accessed from the main
> thread only, but with layers this is possible if some care is taken.
> When forcing the layer to render from a different thread, the implicitly
> created CATransaction has to be flushed explicitly, as we do not have a
> main loop at the end of which it would be flushed.
> 
> We do not force rendering all the time though, as doing that would break
> resize animations given that VLC can not know the right time when display
> refresh will happen, so resizing would look laggy and have glitches, as
> during a resize both the OS and VLC would drive the rendering of the
> layer, resulting in unexpected result.
> To prevent that, when live resizing starts (the user resizing by dragging
> a windows corner), the layer is set into asynchronous rendering mode
> which makes the OS drive the rendering loop completely not only for
> drawing the resize change. While the layer is in asynchronous mode, we
> ignore all update requests from the core, as the layer is anyway updated
> continuously by the OS and forcing rendering from another thread would
> lead to artifacts. Additionally while in live resize, we do not report
> the size changes to the core, as the event takes too long to reach the
> vout Control() function, resulting in the layer content being displayed
> at the wrong (old) size. Instead we take the current viewport size
> as the size and display using that.
> 
> Another unusual thing compared to other vouts is that the VLC OpenGL
> display functions to update the viewport and aspect ratio are not
> called in the Control event handling callback, thats because before
> the render callback is called, the OS sets the OpenGL viewport to match
> the layer backing store size. So setting it in the Control callback
> is useless as it does not make any difference.
> ---
>  modules/video_output/caopengllayer.m | 896 ++++++++++++++++++---------
>  1 file changed, 604 insertions(+), 292 deletions(-)
> 
> diff --git a/modules/video_output/caopengllayer.m 
> b/modules/video_output/caopengllayer.m
> index c6496ed8f0..fb2b2bc02c 100644
> --- a/modules/video_output/caopengllayer.m
> +++ b/modules/video_output/caopengllayer.m
> @@ -8,6 +8,8 @@
>   *          Felix Paul K├╝hne <fkuehne at videolan dot org>
>   *          Pierre d'Herbemont <pdherbemont at videolan dot org>
>   *
> + * Some of the code is based on mpv's video_layer.swift by "der 
> richter"
> + *
>   * This program is free software; you can redistribute it and/or 
> modify it
>   * under the terms of the GNU Lesser General Public License as 
> published by
>   * the Free Software Foundation; either version 2.1 of the License, or
> @@ -35,11 +37,12 @@
>  #include <vlc_plugin.h>
>  #include <vlc_vout_display.h>
>  #include <vlc_opengl.h>
> +#include <vlc_atomic.h>
>  
>  #import <QuartzCore/QuartzCore.h>
>  #import <Cocoa/Cocoa.h>
>  #import <OpenGL/OpenGL.h>
> -#import <dlfcn.h>               /* dlsym */
> +#import <dlfcn.h>
>  
>  #include "opengl/vout_helper.h"
>  
> @@ -54,254 +57,401 @@
>  static void PictureDisplay  (vout_display_t *vd, picture_t *pic, 
> subpicture_t *subpicture);
>  static int Control          (vout_display_t *vd, int query, va_list 
> ap);
>  
> -static void *OurGetProcAddress (vlc_gl_t *gl, const char *name);
> -static int OpenglLock         (vlc_gl_t *gl);
> -static void OpenglUnlock       (vlc_gl_t *gl);
> -static void OpenglSwap         (vlc_gl_t *gl);
> -
> - at protocol VLCCoreAnimationVideoLayerEmbedding <NSObject>
> -- (void)addVoutLayer:(CALayer *)aLayer;
> -- (void)removeVoutLayer:(CALayer *)aLayer;
> -- (CGSize)currentOutputSize;
> +/**
> + * Protocol declaration that drawable-nsobject should follow
> + */
> + at protocol VLCOpenGLVideoViewEmbedding <NSObject>
> +- (void)addVoutSubview:(NSView *)view;
> +- (void)removeVoutSubview:(NSView *)view;
>  @end
>  
> +/**
> + * Layer subclass that handles OpenGL video rendering
> + */
>  @interface VLCCAOpenGLLayer : CAOpenGLLayer
> +{
> +    NSLock *_displayLock;
> +    vout_display_t *_voutDisplay; // All accesses to this must be 
> @synchronized(self)
> +                                  // unless you can be sure it won't 
> be called in teardown
> +    CGLContextObj _glContext;
> +}
>  
> - at property (nonatomic, readwrite) vout_display_t* voutDisplay;
> - at property (nonatomic, readwrite) CGLContextObj glContext;
> -
> +- (instancetype)initWithVoutDisplay:(vout_display_t *)vd;
> +- (void)placePictureWithConfig:(const vout_display_cfg_t *)cfg;
> +- (void)displayFromVout;
> +- (void)reportCurrentLayerSize;
> +- (void)vlcClose;
>  @end
>  
> +/**
> + * View subclass which is backed by a VLCCAOpenGLLayer
> + */
> + at interface VLCVideoLayerView : NSView <CALayerDelegate, 
> NSViewLayerContentScaleDelegate>
> +{
> +    vout_display_t *_vlc_vd; // All accesses to this must be 
> @synchronized(self)
> +}
> +
> +- (instancetype)initWithVoutDisplay:(vout_display_t *)vd;
> +- (void)vlcClose;
> + at end
>  
>  struct vout_display_sys_t {
> +    vout_window_t *embed;
> +    id<VLCOpenGLVideoViewEmbedding> container;
>  
>      picture_pool_t *pool;
>      picture_resource_t resource;
>  
> -    CALayer <VLCCoreAnimationVideoLayerEmbedding> *container;
> -    vout_window_t *embed;
> -    VLCCAOpenGLLayer *cgLayer;
> +    VLCVideoLayerView *videoView; // Layer-backed view that creates videoLayer
> +    VLCCAOpenGLLayer *videoLayer; // Backing layer of videoView
>  
>      vlc_gl_t *gl;
>      vout_display_opengl_t *vgl;
>  
>      vout_display_place_t place;
>  
> -    bool  b_frame_available;
> +    atomic_bool is_ready;
>  };
>  
> -struct gl_sys
> +#pragma mark -
> +#pragma mark OpenGL context helpers
> +
> +/**
> + * Create a new CGLContextObj for use by VLC
> + * This function may try various pixel formats until it finds a 
> suitable/compatible
> + * one that works on the given hardware.
> + * \return CGLContextObj or NULL in case of error
> + */
> +CGLContextObj vlc_CreateCGLContext()
> +{
> +    CGLError err;
> +    GLint npix = 0;
> +    CGLPixelFormatObj pix;
> +    CGLContextObj ctx;
> +
> +    CGLPixelFormatAttribute attribs[12] = {
> +        kCGLPFAAllowOfflineRenderers,
> +        kCGLPFADoubleBuffer,
> +        kCGLPFAAccelerated,
> +        kCGLPFANoRecovery,
> +        kCGLPFAColorSize, 24,
> +        kCGLPFAAlphaSize, 8,
> +        kCGLPFADepthSize, 24,
> +        0, // If ever extending this list, adjust the offset below!
> +        0
> +    };
> +
> +    if (@available(macOS 10.8, *)) {
> +        // Enable automatic graphics switching support, important on 
> Macs
> +        // with dedicated GPUs, as it allows to not always use the 
> dedicated
> +        // GPU which has more power consumption
> +        attribs[10] = kCGLPFASupportsAutomaticGraphicsSwitching;
> +    }
> +
> +    err = CGLChoosePixelFormat(attribs, &pix, &npix);
> +    if (err != kCGLNoError || pix == NULL) {
> +        return NULL;
> +    }
> +
> +    err = CGLCreateContext(pix, NULL, &ctx);
> +    if (err != kCGLNoError || ctx == NULL) {
> +        return NULL;
> +    }
> +
> +    CGLDestroyPixelFormat(pix);
> +    return ctx;
> +}
> +
> +struct vlc_gl_sys
>  {
> -    CGLContextObj locked_ctx;
> -    VLCCAOpenGLLayer *cgLayer;
> +    CGLContextObj cgl; // The CGL context managed by us
> +    CGLContextObj cgl_prev; // The previously current CGL context, if 
> any
>  };
>  
> -/*****************************************************************************
> - * Open: This function allocates and initializes the OpenGL vout 
> method.
> - 
> *****************************************************************************/
> -static int Open (vlc_object_t *p_this)
> +/**
> + * Flush the OpenGL context
> + * In case of double-buffering swaps the back buffer with the front 
> buffer.
> + * \note This function implicitly calls \c glFlush() before it returns.
> + */
> +static void gl_cb_Swap(vlc_gl_t *vlc_gl)
>  {
> -    vout_display_t *vd = (vout_display_t *)p_this;
> -    vout_display_sys_t *sys;
> +    struct vlc_gl_sys *sys = vlc_gl->sys;
> +
> +    // Copies a double-buffered contexts back buffer to front buffer, 
> calling
> +    // glFlush before this is not needed and discouraged for 
> performance reasons.
> +    // An implicit glFlush happens before CGLFlushDrawable returns.
> +    CGLFlushDrawable(sys->cgl);
> +}
> +
> +/**
> + * Make the OpenGL context the current one
> + * Makes the CGL context the current context, if it is not already the 
> current one,
> + * and locks it.
> + */
> +static int gl_cb_MakeCurrent(vlc_gl_t *vlc_gl)
> +{
> +    CGLError err;
> +    struct vlc_gl_sys *sys = vlc_gl->sys;
>  
> -    /* Allocate structure */
> -    vd->sys = sys = calloc(1, sizeof(vout_display_sys_t));
> -    if (sys == NULL)
> +    sys->cgl_prev = CGLGetCurrentContext();
> +
> +    if (sys->cgl_prev != sys->cgl) {
> +        err = CGLSetCurrentContext(sys->cgl);
> +        if (err != kCGLNoError) {
> +            msg_Err(vlc_gl, "Failure setting current CGLContext: %s", 
> CGLErrorString(err));
> +            return VLC_EGENERIC;
> +        }
> +    }
> +
> +    err = CGLLockContext(sys->cgl);
> +    if (err != kCGLNoError) {
> +        msg_Err(vlc_gl, "Failure locking CGLContext: %s", 
> CGLErrorString(err));
>          return VLC_EGENERIC;
> +    }
> +
> +    return VLC_SUCCESS;
> +}
> +
> +/**
> + * Make the OpenGL context no longer current one.
> + * Makes the previous context the current one and unlocks the CGL 
> context.
> + */
> +static void gl_cb_ReleaseCurrent(vlc_gl_t *vlc_gl)
> +{
> +    CGLError err;
> +    struct vlc_gl_sys *sys = vlc_gl->sys;
> +
> +    assert(CGLGetCurrentContext() == sys->cgl);
> +
> +    err = CGLUnlockContext(sys->cgl);
> +    if (err != kCGLNoError) {
> +        msg_Err(vlc_gl, "Failure unlocking CGLContext: %s", 
> CGLErrorString(err));
> +        abort();
> +    }
> +
> +    if (sys->cgl_prev != sys->cgl) {
> +        err = CGLSetCurrentContext(sys->cgl_prev);
> +        if (err != kCGLNoError) {
> +            msg_Err(vlc_gl, "Failure restoring previous CGLContext: 
> %s", CGLErrorString(err));
> +            abort();
> +        }
> +    }
> +
> +    sys->cgl_prev = NULL;
> +}
> +
> +/**
> + * Look up OpenGL symbols by name
> + */
> +static void *gl_cb_GetProcAddress(vlc_gl_t *vlc_gl, const char *name)
> +{
> +    VLC_UNUSED(vlc_gl);
>  
> +    return dlsym(RTLD_DEFAULT, name);
> +}
> +
> +
> +#pragma mark -
> +#pragma mark Module functions
> +
> +static int Open(vlc_object_t *this)
> +{
>      @autoreleasepool {
> +        vout_display_t *vd = (vout_display_t *)this;
> +        vout_display_sys_t *sys;
> +
> +        vd->sys = sys = vlc_obj_calloc(this, 1, sizeof(*sys));
> +        if (sys == NULL)
> +            return VLC_ENOMEM;
> +
> +        // Obtain container NSObject
>          id container = var_CreateGetAddress(vd, "drawable-nsobject");
> -        if (container)
> +        if (container) {
>              vout_display_DeleteWindow(vd, NULL);
> -        else {
> +        } else {
>              sys->embed = vout_display_NewWindow(vd, VOUT_WINDOW_TYPE_NSOBJECT);
>              if (sys->embed)
>                  container = sys->embed->handle.nsobject;
>  
>              if (!container) {
>                  msg_Err(vd, "No drawable-nsobject found!");
> -                goto bailout;
> +                goto error;
>              }
>          }
>  
> -        /* store for later, released in Close() */
> +        // Retain container, released in Close
>          sys->container = [container retain];
> +    
> +        // Create the CGL context
> +        CGLContextObj cgl_ctx = vlc_CreateCGLContext();
> +        if (cgl_ctx == NULL) {
> +            msg_Err(vd, "Failure to create CGL context!");
> +            goto error;
> +        }
>  
> -        [CATransaction begin];
> -        sys->cgLayer = [[VLCCAOpenGLLayer alloc] init];
> -        [sys->cgLayer setVoutDisplay:vd];
> -
> -        [sys->cgLayer performSelectorOnMainThread:@selector(display)
> -                                       withObject:nil
> -                                    waitUntilDone:YES];
> -
> -        if ([container respondsToSelector:@selector(addVoutLayer:)]) {
> -            msg_Dbg(vd, "container implements implicit protocol");
> -            [container addVoutLayer:sys->cgLayer];
> -        } else if ([container 
> respondsToSelector:@selector(addSublayer:)] ||
> -                   [container isKindOfClass:[CALayer class]]) {
> -            msg_Dbg(vd, "container doesn't implement implicit 
> protocol, fallback mode used");
> -            [container addSublayer:sys->cgLayer];
> -        } else {
> -            msg_Err(vd, "Provided NSObject container isn't 
> compatible");
> -            [sys->cgLayer release];
> -            sys->cgLayer = nil;
> -            [CATransaction commit];
> -            goto bailout;
> +        // Create a pseudo-context object which provides needed 
> callbacks
> +        // for VLC to deal with the CGL context. Usually this should 
> be done
> +        // by a proper opengl provider module, but we do not have that 
> currently.
> +        sys->gl = vlc_object_create(vd, sizeof(*sys->gl));
> +        if (unlikely(!sys->gl))
> +            goto error;
> +        
> +        struct vlc_gl_sys *glsys = sys->gl->sys =
> +            vlc_obj_malloc(VLC_OBJECT(sys->gl), sizeof(*glsys));
> +        if (unlikely(!glsys)) {
> +            Close(this);
> +            return VLC_ENOMEM;
> +        }
> +        glsys->cgl = cgl_ctx;
> +        glsys->cgl_prev = NULL;
> +
> +        sys->gl->swap = gl_cb_Swap;
> +        sys->gl->makeCurrent = gl_cb_MakeCurrent;
> +        sys->gl->releaseCurrent = gl_cb_ReleaseCurrent;
> +        sys->gl->getProcAddress = gl_cb_GetProcAddress;
> +
> +        // Set the CGL context to the "macosx-glcontext" as the
> +        // CGL context is needed for CIFilters and the CVPX converter
> +        var_Create(vd->obj.parent, "macosx-glcontext", 
> VLC_VAR_ADDRESS);
> +        var_SetAddress(vd->obj.parent, "macosx-glcontext", cgl_ctx);
> +
> +        dispatch_sync(dispatch_get_main_queue(), ^{
> +            // Create video view
> +            sys->videoView = [[VLCVideoLayerView alloc] 
> initWithVoutDisplay:vd];
> +            sys->videoLayer = (VLCCAOpenGLLayer*)[[sys->videoView 
> layer] retain];
> +            // Add video view to container
> +            if ([container 
> respondsToSelector:@selector(addVoutSubview:)]) {
> +                [container addVoutSubview:sys->videoView];
> +            } else if ([container isKindOfClass:[NSView class]]) {
> +                NSView *containerView = container;
> +                [containerView addSubview:sys->videoView];
> +                [sys->videoView setFrame:containerView.bounds];
> +                [sys->videoLayer reportCurrentLayerSize];
> +            } else {
> +                [sys->videoView release];
> +                [sys->videoLayer release];
> +                sys->videoView = nil;
> +                sys->videoLayer = nil;
> +            }
> +        });
> +        

stray spaces

> +        if (sys->videoView == nil) {
> +            msg_Err(vd,
> +                    "Invalid drawable-nsobject object, must either be 
> an NSView "
> +                    "or comply with the VLCOpenGLVideoViewEmbedding 
> protocol");
> +            goto error;
>          }
> -        [CATransaction commit];
>  
> -        if (!sys->cgLayer)
> -            goto bailout;
> +        // Initialize OpenGL video display
> +        const vlc_fourcc_t *spu_chromas;
>  
> -        if (![sys->cgLayer glContext])
> -            msg_Warn(vd, "we might not have an OpenGL context yet");
> +        if (vlc_gl_MakeCurrent(sys->gl))
> +            goto error;
> +        

stray spaces

> +        sys->vgl = vout_display_opengl_New(&vd->fmt, &spu_chromas, sys->gl,
> +                                           &vd->cfg->viewpoint);
> +        vlc_gl_ReleaseCurrent(sys->gl);
>  
> -        /* Initialize common OpenGL video display */
> -        sys->gl = vlc_object_create(vd, sizeof(*sys->gl));
> -        if (unlikely(!sys->gl))
> -            goto bailout;
> -        sys->gl->makeCurrent = OpenglLock;
> -        sys->gl->releaseCurrent = OpenglUnlock;
> -        sys->gl->swap = OpenglSwap;
> -        sys->gl->getProcAddress = OurGetProcAddress;
> -
> -        struct gl_sys *glsys = sys->gl->sys = malloc(sizeof(*glsys));
> -        if (!sys->gl->sys)
> -            goto bailout;
> -        glsys->locked_ctx = NULL;
> -        glsys->cgLayer = sys->cgLayer;
> -
> -        const vlc_fourcc_t *subpicture_chromas;
> -        video_format_t fmt = vd->fmt;
> -        if (!OpenglLock(sys->gl)) {
> -            sys->vgl = vout_display_opengl_New(&vd->fmt, &subpicture_chromas,
> -                                               sys->gl, &vd->cfg->viewpoint);
> -            OpenglUnlock(sys->gl);
> -        } else
> -            sys->vgl = NULL;
> -        if (!sys->vgl) {
> -            msg_Err(vd, "Error while initializing opengl display.");
> -            goto bailout;
> +        if (sys->vgl == NULL) {
> +            msg_Err(vd, "Error while initializing OpenGL display");
> +            goto error;
>          }
>  
> -        /* setup vout display */
> -        vout_display_info_t info = vd->info;
> -        info.subpicture_chromas = subpicture_chromas;
> -        vd->info = info;
> +        vd->info.has_pictures_invalid = false;
> +        vd->info.subpicture_chromas = spu_chromas;
>  
>          vd->pool    = Pool;
>          vd->prepare = PictureRender;
>          vd->display = PictureDisplay;
>          vd->control = Control;
>  
> -        /* setup initial state */
> -        CGSize outputSize;
> -        if ([container 
> respondsToSelector:@selector(currentOutputSize)])
> -            outputSize = [container currentOutputSize];
> -        else
> -            outputSize = [sys->container visibleRect].size;
> -        vout_display_SendEventDisplaySize(vd, (int)outputSize.width, 
> (int)outputSize.height);
> -
> +        atomic_init(&sys->is_ready, false);
>          return VLC_SUCCESS;
>  
> -    bailout:
> -        Close(p_this);
> +    error:
> +        Close(this);
>          return VLC_EGENERIC;
>      }
>  }
>  
> -static void Close (vlc_object_t *p_this)
> +static void Close(vlc_object_t *p_this)
>  {
>      vout_display_t *vd = (vout_display_t *)p_this;
>      vout_display_sys_t *sys = vd->sys;
>  
> -    if (sys->vgl != NULL && !OpenglLock(sys->gl)) {
> +    atomic_store(&sys->is_ready, false);
> +
> +    if (sys->vgl && !vlc_gl_MakeCurrent(sys->gl)) {
>          vout_display_opengl_Delete(sys->vgl);
> -        OpenglUnlock(sys->gl);
> +        vlc_gl_ReleaseCurrent(sys->gl);
>      }
>  
> -    if (sys->cgLayer) {
> -        if ([sys->container 
> respondsToSelector:@selector(removeVoutLayer:)])
> -            [sys->container removeVoutLayer:sys->cgLayer];
> -        else
> -            [sys->cgLayer removeFromSuperlayer];
> -
> -        if ([sys->cgLayer glContext])
> -            CGLReleaseContext([sys->cgLayer glContext]);
> -
> -        [sys->cgLayer release];
> -    }
> +    [sys->videoView vlcClose];

Should vlcCLose be called before vout_display_opengl_Delete(sys->vgl) ?

It seems that you are using sys->vgl from drawInCGLContext() without checking it.
I'm concerted about this scenario:

MainThread: canDrawInCGLContext() returns YES
Vout thread: Close(): sys->vgl is destroyed, _voutDisplay is set to NULL
MainThread: drawInCGLContext(): use destroyed sys->vgl

One way to fix it is to call [sys->videoView vlcClose]; first and check for _voutDisplay in drawInCGLContext().

> +    dispatch_async(dispatch_get_main_queue(), ^{
> +        // Remove vout subview from container
> +        if ([sys->container 
> respondsToSelector:@selector(removeVoutSubview:)]) {
> +            [sys->container removeVoutSubview:sys->videoView];
> +        }
>  
> -    if (sys->container)
> +        [sys->videoView removeFromSuperview];
> +        [sys->videoView release];
>          [sys->container release];
> +        [sys->videoLayer release];
> +    });
>  
>      if (sys->embed)
>          vout_display_DeleteWindow(vd, sys->embed);
>  
> -    if (sys->gl != NULL)
> -    {
> -        if (sys->gl->sys != NULL)
> -        {
> -            assert(((struct gl_sys *)sys->gl->sys)->locked_ctx == NULL);
> -            free(sys->gl->sys);
> -        }
> +    if (sys->gl) {
> +        struct vlc_gl_sys *glsys = sys->gl->sys;
> +
> +        // It should never happen that the context is destroyed and we
> +        // still have a previous context set, as it would mean non-balanced
> +        // calls to MakeCurrent/ReleaseCurrent.
> +        assert(glsys->cgl_prev == NULL);
> +
> +        CGLReleaseContext(glsys->cgl);
>          vlc_object_release(sys->gl);
>      }

Why are you destroying the window and sys->gl after the dispatch_async call ?

Since the async call can be executed before, during and after these 2 destroys. It feel safer to do the async call at the very end.
That way, you have a more reproducible behavior that won't hide weird bugs.

> -
> -    free(sys);
>  }
>  
> -static picture_pool_t *Pool (vout_display_t *vd, unsigned count)
> +static picture_pool_t *Pool(vout_display_t *vd, unsigned count)
>  {
>      vout_display_sys_t *sys = vd->sys;
>  
> -    if (!sys->pool && !OpenglLock(sys->gl)) {
> +    if (!sys->pool && vlc_gl_MakeCurrent(sys->gl) == VLC_SUCCESS)
> +    {
>          sys->pool = vout_display_opengl_GetPool(sys->vgl, count);
> -        OpenglUnlock(sys->gl);
> -        assert(sys->pool);
> +        vlc_gl_ReleaseCurrent(sys->gl);
>      }
>      return sys->pool;
>  }
>  
> -static void PictureRender (vout_display_t *vd, picture_t *pic, 
> subpicture_t *subpicture)
> +static void PictureRender(vout_display_t *vd, picture_t *pic, 
> subpicture_t *subpicture)
>  {
>      vout_display_sys_t *sys = vd->sys;
>  
> -    if (pic == NULL) {
> -        msg_Warn(vd, "invalid pic, skipping frame");
> -        return;
> -    }
> +    if (vlc_gl_MakeCurrent(sys->gl) == VLC_SUCCESS)
> +    {
> +        vout_display_opengl_Prepare(sys->vgl, pic, subpicture);
> +        vlc_gl_ReleaseCurrent(sys->gl);
>  
> -    @synchronized (sys->cgLayer) {
> -        if (!OpenglLock(sys->gl)) {
> -            vout_display_opengl_Prepare(sys->vgl, pic, subpicture);
> -            OpenglUnlock(sys->gl);
> -        }
> +        atomic_store(&sys->is_ready, true);
>      }
>  }
>  
> -static void PictureDisplay (vout_display_t *vd, picture_t *pic, 
> subpicture_t *subpicture)
> +static void PictureDisplay(vout_display_t *vd, picture_t *pic, 
> subpicture_t *subpicture)
>  {
>      vout_display_sys_t *sys = vd->sys;
>  
> -    @synchronized (sys->cgLayer) {
> -        sys->b_frame_available = YES;
> -
> -        /* Calling display on the non-main thread is not officially 
> supported, but
> -         * its suggested at several places and works fine here. Flush 
> is thread-safe
> -         * and makes sure the picture is actually displayed. */
> -        [sys->cgLayer display];
> -        [CATransaction flush];
> -    }
> +    [sys->videoLayer displayFromVout];
>  
>      picture_Release(pic);
> -
>      if (subpicture)
>          subpicture_Delete(subpicture);
>  }
>  
> -static int Control (vout_display_t *vd, int query, va_list ap)
> +static int Control(vout_display_t *vd, int query, va_list ap)
>  {
>      vout_display_sys_t *sys = vd->sys;
>  
> @@ -318,54 +468,31 @@ static int Control (vout_display_t *vd, int 
> query, va_list ap)
>          {
>              const vout_display_cfg_t *cfg;
>  
> -            if (query == VOUT_DISPLAY_CHANGE_SOURCE_ASPECT || query == 
> VOUT_DISPLAY_CHANGE_SOURCE_CROP) {
> +            if (query == VOUT_DISPLAY_CHANGE_SOURCE_ASPECT ||
> +                query == VOUT_DISPLAY_CHANGE_SOURCE_CROP) {
>                  cfg = vd->cfg;
>              } else {
>                  cfg = (const vout_display_cfg_t*)va_arg (ap, const 
> vout_display_cfg_t *);
>              }
>  
> -            /* we always use our current frame here */
> -            vout_display_cfg_t cfg_tmp = *cfg;
> -            [CATransaction lock];
> -            CGRect bounds = [sys->cgLayer visibleRect];
> -            [CATransaction unlock];
> -            cfg_tmp.display.width = bounds.size.width;
> -            cfg_tmp.display.height = bounds.size.height;
> -
> -            /* Reverse vertical alignment as the GL tex are Y inverted 
> */
> -            if (cfg_tmp.align.vertical == VOUT_DISPLAY_ALIGN_TOP)
> -                cfg_tmp.align.vertical = VOUT_DISPLAY_ALIGN_BOTTOM;
> -            else if (cfg_tmp.align.vertical == 
> VOUT_DISPLAY_ALIGN_BOTTOM)
> -                cfg_tmp.align.vertical = VOUT_DISPLAY_ALIGN_TOP;
> -
> -            vout_display_place_t place;
> -            vout_display_PlacePicture (&place, &vd->source, &cfg_tmp, 
> false);
> -            if (OpenglLock(sys->gl))
> -                return VLC_EGENERIC;
> -
> -            vout_display_opengl_SetWindowAspectRatio(sys->vgl, 
> (float)place.width / place.height);
> -            OpenglUnlock(sys->gl);
> -
> -            sys->place = place;
> +            [sys->videoLayer placePictureWithConfig:cfg];

You seem to call placePictureWithConfig() from the vout thread (Control()) and from MT (drawInCGLContext()), sys->place need to be protected then.

>  
> +            // Note!
> +            // No viewport or aspect ratio is set here, as that needs to be set
> +            // when rendering. The viewport is always set to match the layer
> +            // size by the OS right before the OpenGL render callback, so
> +            // setting it here has no effect.
>              return VLC_SUCCESS;
>          }
>  
>          case VOUT_DISPLAY_CHANGE_VIEWPOINT:
>          {
> -            int ret;
> -
> -            if (OpenglLock(sys->gl))
> -                return VLC_EGENERIC;
> -
> -            ret = vout_display_opengl_SetViewpoint(sys->vgl,
> -                &va_arg (ap, const vout_display_cfg_t* )->viewpoint);
> -            OpenglUnlock(sys->gl);
> -            return ret;
> +            return vout_display_opengl_SetViewpoint(sys->vgl,
> +                        &va_arg (ap, const vout_display_cfg_t* )->viewpoint);
>          }
>  
>          case VOUT_DISPLAY_RESET_PICTURES:
> -            vlc_assert_unreachable ();
> +            vlc_assert_unreachable();
>          default:
>              msg_Err (vd, "Unhandled request %d", query);
>              return VLC_EGENERIC;
> @@ -375,188 +502,373 @@ static int Control (vout_display_t *vd, int 
> query, va_list ap)
>  }
>  
>  #pragma mark -
> -#pragma mark OpenGL callbacks
> +#pragma mark VLCVideoLayerView
>  
> -static int OpenglLock (vlc_gl_t *gl)
> + at implementation VLCVideoLayerView
> +
> +- (instancetype)initWithVoutDisplay:(vout_display_t *)vd
>  {
> -    struct gl_sys *sys = gl->sys;
> -    assert(sys->locked_ctx == NULL);
> +    self = [super init];
> +    if (self) {
> +        _vlc_vd = vd;
>  
> -    CGLContextObj ctx = [sys->cgLayer glContext];
> -    if(!ctx) {
> -        return 1;
> +        self.autoresizingMask = NSViewWidthSizable | NSViewHeightSizable;
> +        self.wantsLayer = YES;
>      }
> +    return self;
> +}
>  
> -    CGLError err = CGLLockContext(ctx);
> -    if (kCGLNoError == err) {
> -        sys->locked_ctx = ctx;
> -        CGLSetCurrentContext(ctx);
> -        return 0;
> +/**
> +* Invalidates VLC objects (notably _vlc_vd)
> +* This method must be called in VLCs module Close (or indirectly by the View)
> +* to ensure all critical VLC resources that might be gone when the module is
> +* closed are properly NULLed. This is necessary as dealloc is only called later
> +* as it has to be done async on the main thread, because NSView must be
> +* dealloc'ed on the main thread and the view own the layer, so the layer
> +* will stay valid until the view is gone, and might still use _vlc_vd
> +* even after the VLC module is gone and the resources would be invalid.
> +*/
> +- (void)vlcClose
> +{
> +    @synchronized (self) {
> +        [(VLCCAOpenGLLayer *)self.layer vlcClose];
> +        _vlc_vd = NULL;
>      }
> -    return 1;
>  }
>  
> -static void OpenglUnlock (vlc_gl_t *gl)
> +- (void)viewWillStartLiveResize
>  {
> -    struct gl_sys *sys = gl->sys;
> -    CGLUnlockContext(sys->locked_ctx);
> -    sys->locked_ctx = NULL;
> +    [(VLCCAOpenGLLayer *)self.layer setAsynchronous:YES];
>  }
>  
> -static void OpenglSwap (vlc_gl_t *gl)
> +- (void)viewDidEndLiveResize
>  {
> -    glFlush();
> +    [(VLCCAOpenGLLayer *)self.layer setAsynchronous:NO];
>  }
>  
> -static void *OurGetProcAddress (vlc_gl_t *gl, const char *name)
> +- (CALayer *)makeBackingLayer
>  {
> -    VLC_UNUSED(gl);
> +    @synchronized(self) {
> +        NSAssert(_vlc_vd != NULL, @"Cannot create backing layer 
> without vout display!");
>  
> -    return dlsym(RTLD_DEFAULT, name);
> +        VLCCAOpenGLLayer *layer = [[VLCCAOpenGLLayer alloc] 
> initWithVoutDisplay:_vlc_vd];
> +        layer.delegate = self;
> +        return [layer autorelease];
> +    }
>  }
>  
> +/* Layer delegate method that ensures the layer always get the
> + * correct contentScale based on whether the view is on a HiDPI
> + * display or not, and when it is moved between displays.
> + */
> +- (BOOL)layer:(CALayer *)layer
> +shouldInheritContentsScale:(CGFloat)newScale
> +   fromWindow:(NSWindow *)window
> +{
> +    return YES;
> +}
> +
> +/*
> + * General properties
> + */
> +
> +- (BOOL)isOpaque
> +{
> +    return YES;
> +}
> +
> +- (BOOL)acceptsFirstResponder
> +{
> +    return YES;
> +}
> +
> +- (BOOL)mouseDownCanMoveWindow
> +{
> +    return NO;
> +}
> +
> +
> +#pragma mark View mouse events
> +
> +/* Left mouse button down */
> +- (void)mouseDown:(NSEvent *)event
> +{
> +    @synchronized(self) {
> +        if (!_vlc_vd) {
> +            [super mouseDown:event];
> +            return;
> +        }
> +
> +        if (event.type == NSLeftMouseDown &&
> +            !(event.modifierFlags & NSControlKeyMask) &&
> +            event.clickCount == 1) {
> +            vout_display_SendEventMousePressed(_vlc_vd, 
> MOUSE_BUTTON_LEFT);
> +        } else {
> +            [super mouseDown:event];
> +        }
> +    }
> +}
> +
> +/* Left mouse button up */
> +- (void)mouseUp:(NSEvent *)event
> +{
> +    @synchronized(self) {
> +        if (!_vlc_vd) {
> +            [super mouseUp:event];
> +            return;
> +        }
> +
> +        if (event.type == NSLeftMouseUp) {
> +            vout_display_SendEventMouseReleased(_vlc_vd, 
> MOUSE_BUTTON_LEFT);
> +        }
> +    }
> +}
> +
> +/* Middle mouse button down */
> +- (void)otherMouseDown:(NSEvent *)event
> +{
> +    @synchronized(self) {
> +        if (_vlc_vd)
> +            vout_display_SendEventMousePressed(_vlc_vd, 
> MOUSE_BUTTON_CENTER);
> +        else
> +            [super otherMouseDown:event];
> +    }
> +}
> +
> +/* Middle mouse button up */
> +- (void)otherMouseUp:(NSEvent *)event
> +{
> +    @synchronized(self) {
> +        if (_vlc_vd)
> +            vout_display_SendEventMouseReleased(_vlc_vd, 
> MOUSE_BUTTON_CENTER);
> +        else
> +            [super otherMouseUp:event];
> +    }
> +}
> +
> +/* Mouse moved */
> +- (void)mouseMoved:(NSEvent *)event
> +{
> +    @synchronized(self) {
> +        if (!_vlc_vd) {
> +            [super mouseMoved:event];
> +            return;
> +        }
> +
> +        vout_display_sys_t *sys = _vlc_vd->sys;
> +
> +        // Convert window-coordinate point to view space
> +        NSPoint pointInView = [self 
> convertPoint:event.locationInWindow fromView:nil];
> +
> +        // Convert to pixels
> +        NSPoint pointInBacking = [self 
> convertPointToBacking:pointInView];
> +
> +        vout_display_SendMouseMovedDisplayCoordinates(_vlc_vd, 
> ORIENT_VFLIPPED,
> +                                                      
> pointInBacking.x, pointInBacking.y,
> +                                                      &sys->place);
> +    }
> +}
> +
> +/* Mouse moved while clicked */
> +- (void)mouseDragged:(NSEvent *)event
> +{
> +    [self mouseMoved:event];
> +}
> +
> +/* Mouse moved while center-clicked */
> +- (void)otherMouseDragged:(NSEvent *)event
> +{
> +    [self mouseMoved:event];
> +}
> +
> +/* Mouse moved while right-clicked */
> +- (void)rightMouseDragged:(NSEvent *)event
> +{
> +    [self mouseMoved:event];
> +}
> +
> + at end
> +
>  #pragma mark -
> -#pragma mark CA layer
> +#pragma mark VLCCAOpenGLLayer
>  
> -/*****************************************************************************
> - * @implementation VLCCAOpenGLLayer
> - *****************************************************************************/
>  @implementation VLCCAOpenGLLayer
>  
> -- (id)init {
> -
> +- (instancetype)initWithVoutDisplay:(vout_display_t *)vd
> +{
>      self = [super init];
>      if (self) {
> +        _displayLock = [[NSLock alloc] init];
> +        _voutDisplay = vd;
> +
> +        struct vlc_gl_sys *glsys = vd->sys->gl->sys;
> +        _glContext = CGLRetainContext(glsys->cgl);
> +
>          [CATransaction lock];
>          self.needsDisplayOnBoundsChange = YES;
>          self.autoresizingMask = kCALayerWidthSizable | kCALayerHeightSizable;
>          self.asynchronous = NO;
> +        self.opaque = 1.0;
> +        self.hidden = NO;
>          [CATransaction unlock];
>      }
>  
>      return self;
>  }
>  
> -- (void)setVoutDisplay:(vout_display_t *)aVd
> +/**
> + * Invalidates VLC objects (notably _voutDisplay)
> + * This method must be called in VLCs module Close (or indirectly by the View).
> + */
> +- (void)vlcClose
>  {
> -    _voutDisplay = aVd;
> +    @synchronized (self) {
> +        _voutDisplay = NULL;
> +    }
>  }
>  
> -- (void)resizeWithOldSuperlayerSize:(CGSize)size
> +- (void)dealloc
>  {
> -    [super resizeWithOldSuperlayerSize: size];
> -
> -    CGSize boundsSize = self.visibleRect.size;
> -
> -    if (_voutDisplay)
> -        vout_display_SendEventDisplaySize(_voutDisplay, 
> boundsSize.width, boundsSize.height);
> +    CGLReleaseContext(_glContext);
> +    [_displayLock release];
> +    [super dealloc];
>  }
>  
> -- (BOOL)canDrawInCGLContext:(CGLContextObj)glContext 
> pixelFormat:(CGLPixelFormatObj)pixelFormat 
> forLayerTime:(CFTimeInterval)timeInterval displayTime:(const 
> CVTimeStamp *)timeStamp
> +- (void)placePictureWithConfig:(const vout_display_cfg_t *)cfg
>  {
> -    /* Only draw the frame if we have a frame that was previously 
> rendered */
> -    if (!_voutDisplay)
> -        return false;
> +    vout_display_sys_t *sys = _voutDisplay->sys;
> +    vout_display_cfg_t tmp_cfg = *cfg;
> +
> +    // Reverse vertical alignment as the GL tex are Y inverted
> +    if (tmp_cfg.align.vertical == VOUT_DISPLAY_ALIGN_TOP)
> +        tmp_cfg.align.vertical = VOUT_DISPLAY_ALIGN_BOTTOM;
> +    else if (tmp_cfg.align.vertical == VOUT_DISPLAY_ALIGN_BOTTOM)
> +        tmp_cfg.align.vertical = VOUT_DISPLAY_ALIGN_TOP;
>  
> -    return _voutDisplay->sys->b_frame_available;
> +    vout_display_PlacePicture(&sys->place, &_voutDisplay->source, 
> &tmp_cfg, false);
>  }
>  
> -- (void)drawInCGLContext:(CGLContextObj)glContext 
> pixelFormat:(CGLPixelFormatObj)pixelFormat 
> forLayerTime:(CFTimeInterval)timeInterval displayTime:(const 
> CVTimeStamp *)timeStamp
> +- (void)resizeWithOldSuperlayerSize:(CGSize)size
>  {
> -    if (!_voutDisplay)
> +    [super resizeWithOldSuperlayerSize:size];
> +
> +    if (self.asynchronous) {
> +        // During live resize, the size is updated in the
> +        // OpenGL draw callback, to ensure atomic size changes
> +        // that are in sync with the real layer size.
> +        // This bypasses the core but is needed for resizing
> +        // without glitches or lags.
>          return;
> -    vout_display_sys_t *sys = _voutDisplay->sys;
> +    }
>  
> -    if (!sys->vgl)
> -        return;
> +    [self reportCurrentLayerSize];
> +}
>  
> -    CGRect bounds = [self visibleRect];
> +- (void)reportCurrentLayerSize
> +{
> +    CGSize newSize = self.visibleRect.size;
> +    CGFloat scale = self.contentsScale;
>  
> -    // x / y are top left corner, but we need the lower left one
> -    vout_display_opengl_Viewport(sys->vgl, sys->place.x,
> -                                 bounds.size.height - (sys->place.y + 
> sys->place.height),
> -                                 sys->place.width, sys->place.height);
> +    // Calculate pixel values
> +    newSize.width *= scale;
> +    newSize.height *= scale;
>  
> -    // flush is also done by this method, no need to call super
> -    vout_display_opengl_Display (sys->vgl, &_voutDisplay->source);
> -    sys->b_frame_available = NO;
> -}
> +    @synchronized(self) {
> +        if (!_voutDisplay)
> +            return;
>  
> --(CGLPixelFormatObj)copyCGLPixelFormatForDisplayMask:(uint32_t)mask
> -{
> -    // The default is fine for this demonstration.
> -    return [super copyCGLPixelFormatForDisplayMask:mask];
> +        vout_display_SendEventDisplaySize(_voutDisplay,
> +                                          newSize.width, newSize.height);
> +    }
>  }
>  
> -- (CGLContextObj)copyCGLContextForPixelFormat:(CGLPixelFormatObj)pixelFormat
> +- (void)display
>  {
> -    // Only one opengl context is allowed for the module lifetime
> -    if(_glContext) {
> -        msg_Dbg(_voutDisplay, "Return existing context: %p", _glContext);
> -        return _glContext;
> -    }
> +    [_displayLock lock];
>  
> -    CGLContextObj context = [super copyCGLContextForPixelFormat:pixelFormat];
> +    [super display];
> +    [CATransaction flush];
>  
> -    // Swap buffers only during the vertical retrace of the monitor.
> -    //http://developer.apple.com/documentation/GraphicsImaging/
> -    //Conceptual/OpenGL/chap5/chapter_5_section_44.html /
> -
> -    GLint params = 1;
> -    CGLSetParameter( CGLGetCurrentContext(), kCGLCPSwapInterval,
> -                     &params );
> +    [_displayLock unlock];
> +}
>  
> -    @synchronized (self) {
> -        _glContext = context;
> +- (void)displayFromVout
> +{
> +    if (self.asynchronous) {
> +        // During live resizing we do not take updates
> +        // from the vout, as those would interfere with
> +        // the rendering currently happening on the main
> +        // thread for the resize. Rendering anyway happens
> +        // triggered by the OS every display refresh, so
> +        // forcing an update here would be useless anyway.
> +        return;
>      }
>  
> -    return context;
> +    [self display];
>  }
>  
> -- (void)releaseCGLContext:(CGLContextObj)glContext
> +- (BOOL)canDrawInCGLContext:(CGLContextObj)glContext
> +                pixelFormat:(CGLPixelFormatObj)pixelFormat
> +               forLayerTime:(CFTimeInterval)timeInterval
> +                displayTime:(const CVTimeStamp *)timeStamp
>  {
> -    // do not release anything here, we do that when closing the module
> +    @synchronized(self) {
> +        if (!_voutDisplay)
> +            return NO;
> +
> +        return (atomic_load(&_voutDisplay->sys->is_ready));
> +    }
>  }
>  
> -- (void)mouseButtonDown:(int)buttonNumber
> +- (void)drawInCGLContext:(CGLContextObj)glContext
> +             pixelFormat:(CGLPixelFormatObj)pixelFormat
> +            forLayerTime:(CFTimeInterval)timeInterval
> +             displayTime:(const CVTimeStamp *)timeStamp
>  {
> -    @synchronized (self) {
> -        if (_voutDisplay) {
> -            if (buttonNumber == 0)
> -                vout_display_SendEventMousePressed (_voutDisplay, 
> MOUSE_BUTTON_LEFT);
> -            else if (buttonNumber == 1)
> -                vout_display_SendEventMousePressed (_voutDisplay, 
> MOUSE_BUTTON_RIGHT);
> -            else
> -                vout_display_SendEventMousePressed (_voutDisplay, 
> MOUSE_BUTTON_CENTER);
> +    vout_display_sys_t *sys = _voutDisplay->sys;
> +
> +    if (vlc_gl_MakeCurrent(sys->gl))
> +        return;
> +
> +    if (self.asynchronous) {
> +        GLint dims[4] = { 0, 0, 0, 0 };
> +        glGetIntegerv(GL_VIEWPORT, dims);
> +        NSSize newSize = NSMakeSize(dims[2], dims[3]);
> +
> +        if (NSEqualSizes(newSize, NSZeroSize)) {
> +            newSize = self.bounds.size;
> +            CGFloat scale = self.contentsScale;
> +            newSize.width *= scale;
> +            newSize.height *= scale;
>          }
> +
> +        vout_display_cfg_t cfg = *_voutDisplay->cfg;
> +
> +        cfg.display.width = newSize.width;
> +        cfg.display.height = newSize.height;
> +
> +        [self placePictureWithConfig:&cfg];
>      }
> +
> +    // Ensure viewport and aspect ratio is correct
> +    vout_display_opengl_Viewport(sys->vgl, sys->place.x, sys->place.y,
> +                                 sys->place.width, sys->place.height);
> +    vout_display_opengl_SetWindowAspectRatio(sys->vgl, 
> (float)sys->place.width / sys->place.height);
> +
> +    vout_display_opengl_Display(sys->vgl, &_voutDisplay->source);
> +    vlc_gl_ReleaseCurrent(sys->gl);
>  }
>  
> -- (void)mouseButtonUp:(int)buttonNumber
> +- (CGLPixelFormatObj)copyCGLPixelFormatForDisplayMask:(uint32_t)mask
>  {
> -    @synchronized (self) {
> -        if (_voutDisplay) {
> -            if (buttonNumber == 0)
> -                vout_display_SendEventMouseReleased (_voutDisplay, 
> MOUSE_BUTTON_LEFT);
> -            else if (buttonNumber == 1)
> -                vout_display_SendEventMouseReleased (_voutDisplay, 
> MOUSE_BUTTON_RIGHT);
> -            else
> -                vout_display_SendEventMouseReleased (_voutDisplay, 
> MOUSE_BUTTON_CENTER);
> -        }
> -    }
> +    CGLPixelFormatObj fmt = CGLGetPixelFormat(_glContext);
> +    
> +    return (fmt) ? CGLRetainPixelFormat(fmt) : NULL;
>  }
>  
> -- (void)mouseMovedToX:(double)xValue Y:(double)yValue
> +- 
> (CGLContextObj)copyCGLContextForPixelFormat:(CGLPixelFormatObj)pixelFormat
>  {
> -    @synchronized (self) {
> -        if (_voutDisplay) {
> -            vout_display_SendMouseMovedDisplayCoordinates 
> (_voutDisplay,
> -                                                           
> ORIENT_NORMAL,
> -                                                           xValue,
> -                                                           yValue,
> -                                                           
> &_voutDisplay->sys->place);
> -        }
> -    }
> +    return CGLRetainContext(_glContext);
>  }
>  
>  @end
> -- 
> 2.21.1 (Apple Git-122.3)
> 
> 
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel



More information about the vlc-devel mailing list