[vlc-devel] Rust work (discussion)

Kartik Ohri kartikohri13 at gmail.com
Sun Sep 20 07:08:28 CEST 2020


Hi!
I appreciate your commendable work and efforts. However, there are no plans
(at least none that I am aware of) to have the complete VLC codebase in C.
Remi has already raised some fair points here
<https://mailman.videolan.org/pipermail/vlc-devel/2020-September/137326.html>
and
here
<https://mailman.videolan.org/pipermail/vlc-devel/2020-September/137325.html>.
In addition, VLC is an enormous project. Trying to port it to Rust without
having some kind of a compatibility layer will stall development of VLC for
some time. This is not beneficial to the end user.

The current plan is to do the error prone stuff in Rust while continuing
the rest of work in C. The ongoing Rust work is the means to enable writing
Rust modules and example Rust modules. As far as I understand, there is
only a small chance that the existing VLC codebase will be moved to Rust.
Some of the stuff where the benefits of having Rust are profound may be
ported but the rest of it will remain as is. Even if such an effort is
undertaken, the timeframe will be long enough that the costs of the
compatibility layer will be amortized and worth it.

At the same time, some of your work may be reused if possible. Thanks.
Regards,
Kartik

On Sun, Sep 20, 2020 at 8:55 AM Lyndon Brown <jnqnfe at gmail.com> wrote:

> Hi all,
>
> There have been at least two blocks of work submitted recently (by
> others) with an aim of starting to enable development of plugins in
> Rust. I feel that I should speak up at this time to create open
> awareness in those working on this of my own efforts in bringing Rust
> to VLC, or is that VLC to Rust...
>
> (Thus far only JBK is aware, who I only just mentioned it to a few
> weeks ago, and even he does not actually know the extent of it).
>
> Specifically, a years-old project to achieve a complete conversion of
> the whole VLC codebase (to Rust).
>
> My intention here is really not to trash/disparage/whatever this other
> effort - it would be great to get Rust into VLC asap - nor seek praise
> for the previously undisclosed effort I've put in myself; simply put, I
> feel it prudent at this point to raise awareness that there are in fact
> now two projects running in parallel, with entirely different
> objectives as to bringing Rust to VLC, and suggest that perhaps, before
> much more effort is put into the the Rust<=>C compatibility layer /
> interface project, it might be a good idea at this stage to give fresh
> thought to how sensible a use of resources that is. If we can assume
> that going full Rust is a reasonable long term goal of VLC, then when
> that occurs, the compatibility layer being worked on now is just going
> to be junked as redundant. Please understand that it makes no
> difference to my own project if this other one continues (I don't want
> anyone to mistakenly think that I'm writing this out of feeling like my
> work is threatened or something), and I can appreciate the
> disappointment felt if your hard work is dismissed; I'm only writing
> this because I worry that the significant effort of putting together
> this Rust<=>C layer for use in the short term (until full Rust) and
> additionally the effort of maintaining it, is just going to turn out to
> not have been worth it. Especially given the possibility that given the
> existence of my project, achievement of a fully Rust VLC might actually
> not be quite as far off from reality as many might have guessed
> (depending upon acceptance by the team, and how fast/slow my project
> progresses of course). If/when VLC goes full Rust, if we even then
> bother having any Rust<=>C compatibility layer, it would naturally be
> one flipped on it's head from that being currently worked on (to allow
> C code to talk to the Rust library), so as I say, the one being worked
> on now would be junked. Is it honestly worth the effort to continue it?
>
> I might as well open up some information about my own project...
>
> My work on this started back in late 2016, when, being a big fan of VLC
> and Rust; thinking VLC to be a great example of something that could
> benefit a lot from Rust; of Rust being rather concretely establishing
> itself as the replacement for C/C++; and wanting a project both to
> learn Rust with and achieve a useful Rust conversion with, I thought
> I'd throw myself at taking a stab at a complete conversion (without any
> prior knowledge of the codebase), and see where that took me.
>
> This was a private project, with the intention only of informing the
> team later on if I managed to achieve enough success and progress with
> it to feel ready to do so, hoping that they would welcome the effort,
> and eventually agree to take it on board to replace the C codebase.
>
> I put a *lot* of effort into it here and there over the next couple of
> years (entirely unfunded), and made some decent progress, but a huge
> amount of work remains to be done.
>
>  - After reorganising the directory structures and renaming the files
> (extensions), I got all of the public and private header contents
> merged into suitable locations, and reorganised some bits and pieces.
>  - I manually applied the most basic syntax changes across the entire
> libvlccore and libvlc libraries, and converted the VLC binaries. (I
> held off making the syntax change to the plugins to make rebasing
> easier in the short term).
>  - I got the entire block of fourcc data, chroma fallback/descriptions
> stuff, including preprocessor, all converted fully into its own crate
> (necessary).
>  - I got a whole bunch of other trivial components of the core fully
> converted irrc, such as esformat stuff, though I can't recall how much.
>  - I got important chunks of the core initialisation stuff done, such
> as command line argument processing, along with logging and including
> help output I seem to recall.
>  - I got the plugin interface and loading system built and working,
> successfully loading and putting to use a converted logger plugin.
>  - I got the clock converted.
>  - I think I had the plugin cache handling done.
>  - I converted a handful of easy plugins.
>  - I hand crafted a complete pair of 'sys' and 'binding' crates for
> talking to PulseAudio from Rust, which I made public and continue to
> maintain, and converted the PulseAudio Aout plugin to use it.
>  - Maybe more that I'm forgetting.
>  - I also spun off hundreds of commits of work for the C codebase, most
> of which I've not yet actually submitted, but I've rebased in the past
> few weeks to do so and have made a start with some of the small items.
>
> All while regularly rebasing on top of new work being pushed everyday
> to the official codebase (I wasn't creating a new Rust codebase from
> scratch, this was a translation).
>
> I thus had it in a state where I could get VLC to load, correctly
> process command line arguments, including those from plugins, output
> help when requested, locate and load plugins, get a logger plugin
> loaded and connected up, and start logging output to it. Although
> unimpressive in the grand picture, still something, and it took a lot
> of effort just to get that far.
>
> Where I left things off:
>  - I was struggling with certain aspects of the playlist code, and was
> pleased to learn that even the team did not actually really understand
> it properly, when it was replaced entirely last year, and I'm eager to
> get back to this project and get that converted now that we have what
> looks to be a sane and understandable playlist implementation. It will
> help tremendously. (and btw guys, there was one part of the new
> solution, possibly the randomiser, I forget now, that I thought was
> just beautiful in it's elegance when I took a look last year -
> awesome!).
>  - I'd done a bunch of work on the core IO pipeline, and was getting
> somewhat close I think to achieving basic flow of raw audio data
> pumping through it. I'm eager to get back to it and achieve that.
>
> I've unfortunately not had the luxury of having been able to work on it
> for more than a year now due to getting side tracked with other stuff,
> but considering the great effort already put in, I'm determined to not
> just let it go to waste if it can be avoided. I'm eager to get back to
> it when I can, (though at the same time I face significant pressure to
> get an income flowing in).
>
> I'm sure that a question in the minds of some at this point might be to
> ask about making my effort thus far public and getting others on board
> to continue the work or collaborate on it with me. While this is
> ultimately what will have to happen, since there's little chance I'd
> manage to get the entire suite of plugins done myself being pragmatic,
> and I'd welcome it then, I've remained reluctant to open it up until
> I've at least got the majority of the core done and maybe a little
> portion of useful output working - I'd just kinda like the satisfaction
> of that at least being a personal achievement first before others step
> in to help cover the remaining plugins.
>
> Another anticipated question might be how I envisioned the full Rust
> solution interacting with other languages:
>  - As stated, the in-tree binaries, along with the entire libvlc and
> libvlccore would be entirely converted to Rust.
>  - At the back end, all plugins would be entirely converted, but would
> continue to interface with existing 3rd-party C libraries (like
> PulseAudio) where applicable and necessary, via thin 'sys' and/or
> 'binding' crates, creating those as necessary, as I did for PulseAudio,
> though some could be switched to Rust based alternatives, and an ideal
> later extension to the project would be to get as many 3rd-party
> libraries also converted as possible to aim for as much userspace code
> being Rust as possible. 3rd-party plugins would just have to get
> onboard with Rust, since it would not be worthwhile I expect to
> additionally try to provide a C API for them; with them either fully
> converting, or splitting their code up into a Rust plugin talking to a
> C library. I would not expect this to be a unreasonable.
>  - On the front end, I have some awareness that there are other users
> of libvlc than the actual VLC binaries, but little knowledge of them,
> and am not in a good position to assess just how much of libvlc and/or
> core would need to be exposed to them to continue to support them via a
> C interface; how easily they might just learn to talk to Rust instead;
> or if it's even worth continuing to support them considering the huge
> benefits of Rust in contrast. Essentially this remains somewhat
> unanswered, but I find the benefits of Rust compelling enough reason to
> push ahead regardless of what would need to be done.
>
> As I write this, I recognise that I'm very tired (long day), so I hope
> I'm not expressing myself too poorly here. I merely mean to raise
> awareness of the existence of this project I started, not come across
> as arrogant or seeking praise or whatever. Sure, I could hold off
> submitting it to reassess tomorrow, but I hope you guys will just take
> my intentions in good faith... (And I hope the team takes no offence to
> my not having engaged with them earlier about this previously secret
> project that I took it upon myself to start working on).
>
> Regards,
> Lyndon
>
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20200920/2fec8c98/attachment.html>


More information about the vlc-devel mailing list