[vlc-devel] [RFC 0/2] New executor API

[Alexandre Janniaux]

Hi,

On Mon, Aug 31, 2020 at 05:17:44PM +0300, Rémi Denis-Courmont wrote:
> Le maanantaina 31. elokuuta 2020, 9.34.12 EEST Steve Lhomme a écrit :
> > > Yes exactly. All the complexity comes from the very disagreeable decision
> > > to have the runner rather than the owner keep track of the tasks. If the
> > > owner keeps track of its tasks, it can interrupt them (due to stop or
> > > time-out) directly, and the tasks can notify their complexition directly
> > > as well.
>
> > IMO that's the runner dealing with what happens to the task is the more
> > generic way, which allow more use cases.
>
> Obviously the runner needs to track what task is waiting to be run. Point is
> that the respective owner of each task should track their pending tasks.
>
> I've never seen a scheduler where task owners just forgot their tasks and let
> the scheduler track the task on behalf of the owners. That misses separation
> of concerns. Besides I can't see how it works - the owner needs to expect the
> result of their queued tasks. For instance, if they're futures, it needs a
> reference to those futures.
>
> > If you tie a task to an owner, it forces the owner to do all kind of things
>
> Like what? And how do you not tie a task to its owner?! At some point you need
> a concrete implementation of something.
>
> The owner needs to know what tasks it's queued. That's just common sense. I've
> seen many task management APIs in C. None really allow the owner to just
> forget about their own "children" tasks.
>
> And typically, delivering the results is one of:
>
> - the owner waits explicitly for completion (no callback involved),
> - the task is tied to the owner and just does the needful (again no callback
> involved),
> - or the above layer has a callback mechanism to deliver the result, not the
> runner. The callback mechanism itself may be a separate abstraction or it may
> just be a type-specific bespoke callback.
>
> (...)

Maybe what people are talking here is just a debate over
thread_pool consumption vs async executor, and we can
simplify the discussion by re-defining the subject, the
target and centering back on the endgoal?

> > One use case we could use in the future is chaining tasks.
>
> There are no reasons why a future abstraction would be tied to the runner
> abstraction. Also what Alexandre already point out...
>
> I however doubt that futures would make much sense in VLC. C cannot provide
> the syntactic sugar to make them nice, nor the coroutines to make them
> efficient. We also don't have a generic mean to wait on multiple futures.

In async client design, many things that are exposed in a
synchronous way actually aren't really and are just here
to make the code less tornado-like. In particular, a wait
on multiple futures is actually only a task that will receive
a status update from multiple other tasks and queue itself
when it receives the correct count of updates.

Thus such primitive is not about finding a generic wait but
instead finding a generic signaling.

I agree that C lacks the sugar for coroutines, meaning that
the state machine probably needs to be handled by hand and
state allocation be done in a more intricate way. But maybe
there's room for this.

The main blocker for me is not really the language itself,
but more that async design is hard to introduce non-locally
and might be somewhat incompatible with sandbox design.

In this way, I like that vlc_executor actually fills a
task since it's easier to spot the scope of the task and
correctly wait for termination of the task, but yet again
maybe scope can be re-asserted.

Regards,
--
Alexandre Janniaux
Videolabs