[vlc-devel] [PATCH] LUA: precompile with luac

[Rafaël Carré]

Le 12-02-15 12:56, Rémi Denis-Courmont a écrit :
> On Wed, 15 Feb 2012 12:46:33 -0500, Rafaël Carré <funman at videolan.org>
> wrote:
>> Le 12-02-15 04:39, John Freed a écrit :
>>> I see the problem but wonder if there might be a better solution. In
>>> particular, as I understand Lua,  execution of a program basically
>>> consists of two parts:
>>> 1) Loading. By this I mean reading the .lua file from disk and
>>> precompiling via luac into bytecode.
>>> 2) Executing. By this I mean running the bytecode in the host's luac
>>> virtual machine.
>>>
>>> Step 2 is exactly the same speed for both lua and luac files. So the
>>> only speed difference is the overhead involved in Step 1.
>>>
>>> We've seen that while standard Lua might produce identical luac files
>>> across 32-bit architectures, and perhaps across 64-bit as well, the
> luac
>>> files differ between 32 and 64 bit. It also seems clear to me that Lua
>>> (because it doesn't offer the ability to create 32-bit luac files using
>>> the 64-bit precompiler) is intended to be used on the host system and
>>> isn't well designed for cross-compiling.
>>>
>>> One solution is to fork Lua.
>>
>> Yeah that way is fine but we forked only loading code, not the tool to
>> build .luac files.
> 
> Taht's not true. We forked both the loader and the compiler. You cannot
> use contribs to build a CROSS-compiler (Lua or otherwise); that is not new
> and not specific to Lua.

Hm that's right.

It's still a problem for Windows because it's typically cross-compiled only.

>> Rémi refuses to mix host & build tools in the contrib, but doesn't
>> acknowledge that he didn't fork the necessary luac build tool.
> 
> I do not refuse to do it. However, I have not seen any sane proposal to
> that end, and I cannot think of any.
> 
>>> Another, it seems to me, is to load *time critical* lua files when VLC
>>> starts. These could be placed in a special lua/init directory. By
>>> performing Step #1 above during VLC startup, the speed issues noted by
>>> Rémi disappear.
>>
>> There is no time critical lua file; the compile times I benchmarked and
>> mentioned are easily lost in the noise.
> 
> There are. The playlist parsers are involved in many cases while probing
> the input format. The SD plugins are all run when enumerating services.
> Moreover it turns out that most of the Lua scripts belong in one of these
> two categories.

Still the additional CPU load is minimal.

I was thinking of distributing lua files, and compiling them once per
run then caching the bytecode, thus no .luac required.

Or then again compile them at installation.

No idea how hard are those 2 solutions

>>> I think it's tremendously advantageous to distribute lua files rather
>>> than luac files. For one, it encourages people to write their own, and
>>> to make minor local modifications where necessary. Neither is feasible
>>> with luac files.
>>
>> Yeah this is an advantage even if those lua files normally are in an
>> admin owned folder.
> 
> You can download the original Lua files from vlc.git or from the tarballs
> (and run them). I do not see any tremendousness here.

It's not particularly difficult but having plain text files in our
distributions is nicer, not to mention smaller to distribute.

>>> For another, it avoids the problem of either forking Lua or waiting for
>>> upstream to make fixes.
>>
>> There is no upstream intention to make luac output a cross platform
>> format according to discussion on #lua channel.
> 
> That's not a problem with our contribs.
> 
> And distros are free to follow upstream or VLC Lua. They should not
> mismatch the compiler and the library (though that would also work in some
> cases).
> 
>> That micro optimization just makes packaging difficult,
> 
> It has been in previous releases, and it has not caused nay problems.

Yeah I mostly discovered it because of win64 buildbot which is built on
x86_64.
It'll be a problem for people who build win32 on x86_64 (i.e. not our
buildbot setup but might affect j-b when he makes a release binary)