<html dir="ltr"><head></head><body style="text-align:left; direction:ltr;"><div>Hi Kartik,</div><div><br></div><div>Yes, I've noticed no prior discussion either about any such official plans for a full conversion; only exploratory interest at this stage in enabling Rust in plugins in the GSoC projects page (I think it was), as you are pursuing, which at least acknowledges interest in Rust. It's understandable for the project to at least start there. But that did not deter my choosing to see what I could achieve.</div><div><br></div><div>I don't read everything on the ML, but I did see Remi's recent comment you linked to about himself and a few others not yet having found sufficient time to learn Rust.</div><div><br></div><div>Given this is C, there's pretty wide scope for what can be considered "error prone stuff" of course and what would benefit from Rust.</div><div><br></div><div>Although there are some clear positives to the incremental approach and I certainly appreciate the enormous size of the codebase, a large endeavour like this doesn't tend to phase me if I can see a realistic prospect of accomplishing it given enough time, and should I believe the result is worth it. I can be rather tenacious about throwing my all at big tasks like this and pushing through the workload. Although I've been side tracked here and there, and have a serious income problem, I've remained adamant that eventually, sooner or later, I'll manage to get there.</div><div><br></div><div>In terms of stalling development, that would of course be undesirable, but inevitably no matter how VLC switches to Rust, assuming in the end it will fully switch, whether that's all in one go or in small pieces, it's going to eat up a roughly equal amount of the team's time in terms of reviewing the conversions, and thus a roughly equal total impact on stalling other enhancements over the relevant time period. Taking an incremental approach inevitably gets some Rust into end-user hands sooner (which is great), but yet on top of the time reviewing plugin conversions, additional team time needs to be spent reviewing all of the compatibility layer code (and possibly participating in its development), chasing any bugs with it, and having to spend time adjusting it to reflect applicable changes in the C interfaces over time and testing those change, for as long as it exists (until a full conversion is completed). Which is additional time stalling other enhancements, even if it is broken up over time. This doesn't apply to the all-in-one approach.</div><div><br></div><div>If the team were to take a big break at some point on current work to refocus on a full Rust conversion, then of course that would certainly stall the subsequent release significantly, but that's not been my intention; it is definitely right for the 4.0 release to happen first, at least, and my intention has been to get as much conversion work done as possible in parallel to the C development, with minimal participation of the team until review, largely for the very reason of avoiding stalling their continuous work on other enhancements. The later hypothetical big review and switchover I would expect to be suitably planned and co-ordinated with the team such as to minimise disruption and stalling as much as possible. Of course on the other hand, my approach does involve the burden of significant rebasing work to keep in sync with the C work, which I've been unavoidably shouldering as an accepted consequence of my choice to pursue this approach, and of course, as I mentioned, financially supporting my project is an issue.</div><div><br></div><div>(I hope I'm not repeating myself too much).</div><div><br></div><div>Of course this is just my opinion and I recognise that there are positives and negative to both approaches. Regardless of whatever choice you make about about continuing to pursue the compatibility layer, and I'm sure I've done nothing to dissuade you, I'm pleased to see additional interest in getting Rust into VLC, and wish you luck in your efforts. Whether or not we can agree on a compatibility layer being worthwhile for enabling Rust in the short term, we both want Rust; can agree that getting it sooner is better than later; and want to see plugins getting converted, so at the very least, we will both be complementing each other's work in that area I'm sure. :)</div><div><br></div><div>Regards,</div><div>Lyndon</div><div><br></div><div>On Sun, 2020-09-20 at 10:38 +0530, Kartik Ohri wrote:</div><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div dir="ltr">Hi!<div>I appreciate your commendable work and efforts. However, there are no plans (at least none that I am aware of) to have the complete VLC codebase in C. Remi has already raised some fair points <a href="https://mailman.videolan.org/pipermail/vlc-devel/2020-September/137326.html">here</a> and <a href="https://mailman.videolan.org/pipermail/vlc-devel/2020-September/137325.html">here</a>. In addition, VLC is an enormous project. Trying to port it to Rust without having some kind of a compatibility layer will stall development of VLC for some time. This is not beneficial to the end user.</div><div><br></div><div>The current plan is to do the error prone stuff in Rust while continuing the rest of work in C. The ongoing Rust work is the means to enable writing Rust modules and example Rust modules. As far as I understand, there is only a small chance that the existing VLC codebase will be moved to Rust. Some of the stuff where the benefits of having Rust are profound may be ported but the rest of it will remain as is. Even if such an effort is undertaken, the timeframe will be long enough that the costs of the compatibility layer will be amortized and worth it.</div><div><br></div><div>At the same time, some of your work may be reused if possible. Thanks.</div><div>Regards,</div><div>Kartik</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Sep 20, 2020 at 8:55 AM Lyndon Brown <<a href="mailto:jnqnfe@gmail.com">jnqnfe@gmail.com</a>> wrote:<br></div><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">Hi all,<br>
<br>
There have been at least two blocks of work submitted recently (by<br>
others) with an aim of starting to enable development of plugins in<br>
Rust. I feel that I should speak up at this time to create open<br>
awareness in those working on this of my own efforts in bringing Rust<br>
to VLC, or is that VLC to Rust...<br>
<br>
(Thus far only JBK is aware, who I only just mentioned it to a few<br>
weeks ago, and even he does not actually know the extent of it).<br>
<br>
Specifically, a years-old project to achieve a complete conversion of<br>
the whole VLC codebase (to Rust).<br>
<br>
My intention here is really not to trash/disparage/whatever this other<br>
effort - it would be great to get Rust into VLC asap - nor seek praise<br>
for the previously undisclosed effort I've put in myself; simply put, I<br>
feel it prudent at this point to raise awareness that there are in fact<br>
now two projects running in parallel, with entirely different<br>
objectives as to bringing Rust to VLC, and suggest that perhaps, before<br>
much more effort is put into the the Rust<=>C compatibility layer /<br>
interface project, it might be a good idea at this stage to give fresh<br>
thought to how sensible a use of resources that is. If we can assume<br>
that going full Rust is a reasonable long term goal of VLC, then when<br>
that occurs, the compatibility layer being worked on now is just going<br>
to be junked as redundant. Please understand that it makes no<br>
difference to my own project if this other one continues (I don't want<br>
anyone to mistakenly think that I'm writing this out of feeling like my<br>
work is threatened or something), and I can appreciate the<br>
disappointment felt if your hard work is dismissed; I'm only writing<br>
this because I worry that the significant effort of putting together<br>
this Rust<=>C layer for use in the short term (until full Rust) and<br>
additionally the effort of maintaining it, is just going to turn out to<br>
not have been worth it. Especially given the possibility that given the<br>
existence of my project, achievement of a fully Rust VLC might actually<br>
not be quite as far off from reality as many might have guessed<br>
(depending upon acceptance by the team, and how fast/slow my project<br>
progresses of course). If/when VLC goes full Rust, if we even then<br>
bother having any Rust<=>C compatibility layer, it would naturally be<br>
one flipped on it's head from that being currently worked on (to allow<br>
C code to talk to the Rust library), so as I say, the one being worked<br>
on now would be junked. Is it honestly worth the effort to continue it?<br>
<br>
I might as well open up some information about my own project...<br>
<br>
My work on this started back in late 2016, when, being a big fan of VLC<br>
and Rust; thinking VLC to be a great example of something that could<br>
benefit a lot from Rust; of Rust being rather concretely establishing<br>
itself as the replacement for C/C++; and wanting a project both to<br>
learn Rust with and achieve a useful Rust conversion with, I thought<br>
I'd throw myself at taking a stab at a complete conversion (without any<br>
prior knowledge of the codebase), and see where that took me.<br>
<br>
This was a private project, with the intention only of informing the<br>
team later on if I managed to achieve enough success and progress with<br>
it to feel ready to do so, hoping that they would welcome the effort,<br>
and eventually agree to take it on board to replace the C codebase.<br>
<br>
I put a *lot* of effort into it here and there over the next couple of<br>
years (entirely unfunded), and made some decent progress, but a huge<br>
amount of work remains to be done.<br>
<br>
- After reorganising the directory structures and renaming the files<br>
(extensions), I got all of the public and private header contents<br>
merged into suitable locations, and reorganised some bits and pieces.<br>
- I manually applied the most basic syntax changes across the entire<br>
libvlccore and libvlc libraries, and converted the VLC binaries. (I<br>
held off making the syntax change to the plugins to make rebasing<br>
easier in the short term).<br>
- I got the entire block of fourcc data, chroma fallback/descriptions<br>
stuff, including preprocessor, all converted fully into its own crate<br>
(necessary).<br>
- I got a whole bunch of other trivial components of the core fully<br>
converted irrc, such as esformat stuff, though I can't recall how much.<br>
- I got important chunks of the core initialisation stuff done, such<br>
as command line argument processing, along with logging and including<br>
help output I seem to recall.<br>
- I got the plugin interface and loading system built and working,<br>
successfully loading and putting to use a converted logger plugin.<br>
- I got the clock converted.<br>
- I think I had the plugin cache handling done.<br>
- I converted a handful of easy plugins.<br>
- I hand crafted a complete pair of 'sys' and 'binding' crates for<br>
talking to PulseAudio from Rust, which I made public and continue to<br>
maintain, and converted the PulseAudio Aout plugin to use it.<br>
- Maybe more that I'm forgetting.<br>
- I also spun off hundreds of commits of work for the C codebase, most<br>
of which I've not yet actually submitted, but I've rebased in the past<br>
few weeks to do so and have made a start with some of the small items.<br>
<br>
All while regularly rebasing on top of new work being pushed everyday<br>
to the official codebase (I wasn't creating a new Rust codebase from<br>
scratch, this was a translation).<br>
<br>
I thus had it in a state where I could get VLC to load, correctly<br>
process command line arguments, including those from plugins, output<br>
help when requested, locate and load plugins, get a logger plugin<br>
loaded and connected up, and start logging output to it. Although<br>
unimpressive in the grand picture, still something, and it took a lot<br>
of effort just to get that far.<br>
<br>
Where I left things off:<br>
- I was struggling with certain aspects of the playlist code, and was<br>
pleased to learn that even the team did not actually really understand<br>
it properly, when it was replaced entirely last year, and I'm eager to<br>
get back to this project and get that converted now that we have what<br>
looks to be a sane and understandable playlist implementation. It will<br>
help tremendously. (and btw guys, there was one part of the new<br>
solution, possibly the randomiser, I forget now, that I thought was<br>
just beautiful in it's elegance when I took a look last year -<br>
awesome!).<br>
- I'd done a bunch of work on the core IO pipeline, and was getting<br>
somewhat close I think to achieving basic flow of raw audio data<br>
pumping through it. I'm eager to get back to it and achieve that.<br>
<br>
I've unfortunately not had the luxury of having been able to work on it<br>
for more than a year now due to getting side tracked with other stuff,<br>
but considering the great effort already put in, I'm determined to not<br>
just let it go to waste if it can be avoided. I'm eager to get back to<br>
it when I can, (though at the same time I face significant pressure to<br>
get an income flowing in).<br>
<br>
I'm sure that a question in the minds of some at this point might be to<br>
ask about making my effort thus far public and getting others on board<br>
to continue the work or collaborate on it with me. While this is<br>
ultimately what will have to happen, since there's little chance I'd<br>
manage to get the entire suite of plugins done myself being pragmatic,<br>
and I'd welcome it then, I've remained reluctant to open it up until<br>
I've at least got the majority of the core done and maybe a little<br>
portion of useful output working - I'd just kinda like the satisfaction<br>
of that at least being a personal achievement first before others step<br>
in to help cover the remaining plugins.<br>
<br>
Another anticipated question might be how I envisioned the full Rust<br>
solution interacting with other languages:<br>
- As stated, the in-tree binaries, along with the entire libvlc and<br>
libvlccore would be entirely converted to Rust.<br>
- At the back end, all plugins would be entirely converted, but would<br>
continue to interface with existing 3rd-party C libraries (like<br>
PulseAudio) where applicable and necessary, via thin 'sys' and/or<br>
'binding' crates, creating those as necessary, as I did for PulseAudio,<br>
though some could be switched to Rust based alternatives, and an ideal<br>
later extension to the project would be to get as many 3rd-party<br>
libraries also converted as possible to aim for as much userspace code<br>
being Rust as possible. 3rd-party plugins would just have to get<br>
onboard with Rust, since it would not be worthwhile I expect to<br>
additionally try to provide a C API for them; with them either fully<br>
converting, or splitting their code up into a Rust plugin talking to a<br>
C library. I would not expect this to be a unreasonable.<br>
- On the front end, I have some awareness that there are other users<br>
of libvlc than the actual VLC binaries, but little knowledge of them,<br>
and am not in a good position to assess just how much of libvlc and/or<br>
core would need to be exposed to them to continue to support them via a<br>
C interface; how easily they might just learn to talk to Rust instead;<br>
or if it's even worth continuing to support them considering the huge<br>
benefits of Rust in contrast. Essentially this remains somewhat<br>
unanswered, but I find the benefits of Rust compelling enough reason to<br>
push ahead regardless of what would need to be done.<br>
<br>
As I write this, I recognise that I'm very tired (long day), so I hope<br>
I'm not expressing myself too poorly here. I merely mean to raise<br>
awareness of the existence of this project I started, not come across<br>
as arrogant or seeking praise or whatever. Sure, I could hold off<br>
submitting it to reassess tomorrow, but I hope you guys will just take<br>
my intentions in good faith... (And I hope the team takes no offence to<br>
my not having engaged with them earlier about this previously secret<br>
project that I took it upon myself to start working on).<br>
<br>
Regards,<br>
Lyndon<br>
<br>
_______________________________________________<br>
vlc-devel mailing list<br>
To unsubscribe or modify your subscription options:<br>
<a href="https://mailman.videolan.org/listinfo/vlc-devel" rel="noreferrer" target="_blank">https://mailman.videolan.org/listinfo/vlc-devel</a></blockquote></div>
<pre>_______________________________________________</pre><pre>vlc-devel mailing list</pre><pre>To unsubscribe or modify your subscription options:</pre><a href="https://mailman.videolan.org/listinfo/vlc-devel"><pre>https://mailman.videolan.org/listinfo/vlc-devel</pre></a></blockquote>
</body></html>