[vlc-devel] VLC Plugin for Mac OS X (Lion & upwards): some progress, but further help requested

Fri May 11 16:44:00 CEST 2012

Hello all,

For a while now, I've been working at getting the VLC browser plugin working on Mac OS X again, in particular Mac OS X Lion & modern browsers. I've been having some success, but I think I'll some help getting the thing finalized, because I am lacking some key knowledge.

- First some introductory information about browser plugins on the mac:

The NPAPI for mac allows for different so-called "Drawing models". In each of these drawing models, the contract  for drawing graphical output to the plugin embedded zone in the browser page, is different. The latest VLC Plugin binary for mac (1.0.5) used the "Carbon" drawing model, which in modern browsers is hardly supported anymore. (Safari refuses it outright, Firefox requires you to "reboot in 32-bit mode"). That's the reason users are having so much trouble using that plugin.

The more modern drawing models are called "CoreGraphics" drawing model and "CoreAnimation" drawing model. Neither allow direct manipulation of the browser window anymore, but provide abstract "drawing contexts" (either Core Graphics or Core Animation contexts, depending on the drawing model chosen). A plugin chooses which drawing model it would like to use by making appropriate NPN calls in NPP_New() (The official terminology is "drawing model negotiation")

Modern browsers also run the plugin (which is just a shared library) inside a separate process from the browser process (to prevent plugin crashes causing browser crashes). This means that a 64-bit browser binary can launch either a 32-bit plugin or a 64-bit plugin, PROVIDED it uses a modern drawing method. (Indeed 64-bit Firefox requires you top "reboot in 32-bit mode" for a 32-bit "Carbon" drawing model plugin, but runs a 32-bit "CoreGraphics" or "CoreAnimation" drawing model plugin without needing such a reboot. Safari 64-bit also launches such 32-bit plugins without issues.

32-bit browsers on the other hand cannot launch 64-bit plugins. Examples include Google Chrome and Opera, which are still distributed as 32-bit binaries on the mac. 

Considering all the above, the most compatible architecture for a modern mac would seem to be either a "CoreGraphics" or a "CoreAnimation" 32-bit plugin, since basically all modern browsers out there will load that.

- The minimalist approach

With the above in mind, I set out to build just such a plugin (32-bit CoreGraphics drawing model) by tweaking vlc's minimal_macosx module, and npapi-vlc just enough to make it work. This worked quite well, the diffs against the current git repositories are:

- vlc project:
https://github.com/jamesbates/vlc/commit/a0b5ff19a088769a7b2dfebd8d49f83366947a1e
https://github.com/jamesbates/vlc/commit/7529cc398425319b8682961c4c7087ff4e0622c3

- npapi-vlc project:
https://github.com/jamesbates/npapi-vlc/commit/518330285425037807264ff797c8cf3c8e399f97
https://github.com/jamesbates/npapi-vlc/commit/7a8e62733977999a7c946706313aa510670583f1
https://github.com/jamesbates/npapi-vlc/commit/b2506d97b8f0ea24b958f2700ed4d0bd0f4c37ac
https://github.com/jamesbates/npapi-vlc/commit/57d9a7ff47eeabb9b62abb5b4d5c9ea8a9d6db01

I've provided the resulting binary here: https://ssl.jamesbates.de/vlc/VLC-Plugin-2.1.0-darwin11-i686.tar.bz2 and you are welcome to try it out (This isn't an installer, it's just a tar.bz2 of what should go in /Library/Internet Plug-Ins/ on Mac OS X, and inteded for developer testing only (lots of debug output)).

Now the main issue with this (as with the original plugin), is that the video appears in its own window, outside the browser. As far as I can tell from the code, this has always been the case of the Mac version of the VLC Plugin.

- Getting the video embedded in the browser window

This is where I need help: basically, I want to use the exisiting OpenGL display machinery already present in vlc, and get that into the embedded browser view. Googling through the Internet, it seems pretty clear that this is totally impossible with the CoreGraphics drawing model, and the CoreAnimation drawing model is needed. In particular there is a CAOpenGLLayer provided by CoreAnimation which seems to be indicated for the job.

And so I went ahead, and tried to get it implemented. The result so far is here: https://github.com/jamesbates/npapi-vlc.git, branch macosx (don't forget to do  git checkout macosx after clone-ing. master is just a copy of the official repo). Current status is:

- in Safari and in Google Chrome, video actually appears inside the plugin area (yay!!) but it isn't very stable: it shows quite haltingly, and as soon as you do anything with the browser window (click, drag, focus), the video stops.
- In Firefox the video doesn't show at all
- In Opera, the plugin doesn't even load (dunno why yet)

The main problem is that vlc video playback is basically a "push" model: a vout module should react to and implement pictureRender and pictureDisplay methods, which libvlccore calls at its discression. Core Animation on the other hand uses a "pull" model, and expects the implementing layer to render content whenever IT calls you. You can "ping" core animation and say: "hey, please call my update method" (CAOpenGlLaye:: setNeedsDisplay), and then it will call your display method "at some point in the future" (i.e. asynchronously).

I attempt to marry the two using some complicated thread sycnrhonization logic, that you'll find in npapi-vlc/npapi/macosx/plugins/caopengl_vout.m, but it's ugly and obviously not very stable (I suspect deadlocks &c. as the reason for video playback halting, or sometimes not starting).

So if anyone out there knows more about the OpenGL stuff in VLC, or about Core Animation on the mac, or about the threading libraries in VLC (basically just #define's around libpthread as far as I can tell), and could take a look, this would be very much appreciated!

Any other strategy for getting the video playback into the plugin area would obviously also be welcome!

Regards,
James Bates

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4364 bytes
Desc: not available
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20120511/9b264b07/attachment.bin>