[vlc-devel] [PATCH] 64-bit support for mozilla plugin on Mac OS X

[David Menestrina]

This patchset (against the 1.1 fork) fixes the VLC mozilla plugin to work
in 64 bit browsers on Mac OS X (currently the only such browser is Safari).
64-bit browser plugins can no longer use the old Carbon based QuickDraw APIs.
Instead, they must implement either the Core Graphics or Core Animation drawing
models.  Further, they must implement the Cocoa event model.  Here, we create a
new vout display module that writes to a Core Animation layer, and then we use this
vout display to implement the Core Animation drawing model.

I am quite new to both VLC and Mac OS X development, so there are bound to be bugs,
and there are even a few known issues that I just don't have the expertise to figure
out at this time.  I would greatly appreciate the assistance of some real Mac OS
developers to look over the code.  Known issues:
  1) Cocoa documentation says that NSView is not threadsafe, yet this
     code uses NSView in the vout thread for entering fullscreen mode.
  2) Perhaps due to some release/retain issues, or perhaps some race
     conditions, the plugin crashes when closing the page using it.  The
     stack trace for the crash is unfortunately in Safari code, which
     makes it difficult to debug.
  3) I have read that Cocoa requires an NSAutoreleasePool to exist for
     any thread that makes Cocoa calls.  The vout thread does not have one,
     and therefore the plugin will leak memory.
  4) (a simple one to fix) Very few events are handled in fullscreen mode,
     and in fact they are not implemented properly.  (The vout should simply
     pass the events on to VLC rather than take action on its own.)

The following is a rundown of the individual patches involved:
[PATCH 1/5] Select wrapper video output if --vout is specified on Mac OS
In VLC 1.1, Mac OS X still uses video output modules instead of vout display
modules via vout_wrapper.  The plugin needs to select the calayer vout
display module to work properly.  Therefore, if --vout is specified, then
we choose vout_wrapper as the video_output module.

[PATCH 2/5] Change plugin path for Mozilla plugin on Mac OS X
This is also a patch that only seems necessary in the 1.1 branch.  It's just
about setting the path where the Mozilla plugin finds the VLC plugins.

[PATCH 3/5] Add CALayer module
(i.e. the calayer vout display module)

[PATCH 4/5] Add coreanim.m for Core Animation code that must use Objective C.
The Core Animation API can only be accessed in Objective C, and the mozilla
plugin previously had no need for Objective C.  This patch adds the file for
Objective C support of core animation, and also sets up libtool to properly
build Objective C in the plugins directory.

[PATCH 5/5] Use Core Animation drawing model, Cocoa event model in 64 bit plugin.
Updates the actual mozilla plugin code to use the Core Animation drawing model.
*Only* when compiled for 64-bit.  The 32 bit plugin will still use QuickDraw,
even if the browser supports Core Animation.

This work was done by Veetle, who are very interested in giving back to
the community to improve various parts of VLC.