[vlc-devel] Implementing crop using the current vout API

[Felix Paul Kühne]

Hello guys,

as some of you might have seen on the forums or trac, cropping the video isn't supported in the 2.0.0 release of VLC for the Mac, mostly because I didn't know prior to release that the video output module needs to implement the functionality, since it isn't provided by the core (yet? at least, I get a "TODO FIXME" error message related to core crop when blocking crop events in the vout).

Anyway, based upon the x11 vout, I cooked up a patch which provides the needed functionality quite nicely:

http://pastie.org/3565475

However, this code does only work for a certain set of chroma formats and video sizes [1]. Otherwise, I get repeatable crashes in swscale [2], the OS graphics driver [3] and the core's video filter functionality [4].

Additionally, when changing the crop border, my debug log is spammed with error messages from the core about the use of "ThreadExecuteCropBorder". So I guess, this isn't the intended way to change the crop border? Or is this just a debug left over?
Furthermore, there are artifacts when cropping 10bit content. [5]

It would be really cool to get some advice on this issue, since I'm a bit stuck at the moment.


Thanks,

Felix


[1] basically, the only working samples I got supporting all crop formats are H.264 (baseline profile) videos with Planar 4:2:0 YUV with a 16:9 AR. A native AR of 16:10 leads to the mentioned artifacts or crashes.
[2] http://pastie.org/3565494
[3] http://pastie.org/3565509
[4] http://pastie.org/3565537
[5] http://cl.ly/3P3G3a1P3U431K3d212K and http://cl.ly/2E3Z1d382s163n3i2B3u