[vlc-devel] [RFC 0/2] New executor API

[Alexandre Janniaux]

Hi,

On Mon, Aug 31, 2020 at 08:34:12AM +0200, Steve Lhomme wrote:
> On 2020-08-28 15:51, Rémi Denis-Courmont wrote:
> > Le perjantaina 28. elokuuta 2020, 11.25.16 EEST Alexandre Janniaux a écrit :
> > > > > > Besides, timing out is an error that deserves logging. Interrupt is
> > > > > > not.
> > > > > > So using the same callback for both purpose looks dubious to me.
> > > > >
> > > > > The goal is simplicity.
> > > >
> > > > By making things more complicated. Yeah right...
> > >
> > > I agree with Remi, if you want simplicity, the best thing
> > > would be to limit the asynchronous framework to the
> > > asynchronous execution and signalling like almost every other
> > > asynchronous framework.
> > >
> > > Especially since youǘe talked about some `future` object, and
> > > thus a pro-actor model which doesn't have ways to do
> > > signalling from the outside of the asynchronous framework to
> > > the task by itself in almost all implementation, and instead
> > > have each layer of promise being able to return either a result
> > > or an error that the future will catch, and an internal
> > > composition rule for each layer's result.
> > >
> > > In the case the vlc_executor is killed, the preparser client
> > > needs to be killed first since it is using the executor, and
> > > thus can cancel its tasks internally directly, which might be
> > > at the middle of the preprocessing, at the beginning or right
> > > before the end. It's much less convoluted than having the
> > > preparser client use the vlc_executor to cancel the taks, which
> > > will just really call the preparser specific cancellation code
> > > anyway.
> >
> > Yes exactly. All the complexity comes from the very disagreeable decision to
> > have the runner rather than the owner keep track of the tasks. If the owner
> > keeps track of its tasks, it can interrupt them (due to stop or time-out)
> > directly, and the tasks can notify their complexition directly as well.
>
> IMO that's the runner dealing with what happens to the task is the more
> generic way, which allow more use cases. If you tie a task to an owner, it
> forces the owner to do all kind of things when in fact it didn't want to
> care at all what happens to the tasks it starts. (ie the finished callback
> should have all it needs to deal with the result/timeout/error/cancelation).
> Sure you may want to cancel things, but there are probably cases you don't
> want to. If the framework forces you to deal with the state of the tasks you
> launch (which may all be very different from each other) that's a lot of
> complexity added where it can be isolated in the task to run.
>
> One use case we could use in the future is chaining tasks. When task A
> finishes, it does some things and start task B. When task B finishes, it
> does some things and start task C. etc. If you put the owner in charge of
> the state of each task that becomes a big mess.
>
> Chaining task could be interesting to do async I/O. You launch a task to
> read/write and when it's do it forwards the data wherever it needs to and
> chain the next same task. You don't need a "manager" to handle it.

The way you do this in usual async code is by letting the task
that finished start the second task that would depend on it.

IMHO that's a much simpler design because:

 - it's pretty straightforward.

 - you don't have to write a directed acyclic graph to apply
   a topological sort and know what you can execute, and
   potentially define a policy for scheduling.

 - it's yet again an API entrypoint less, simplifying the API

 - it could be written with generators and state machine,
   which is litterally how you do to write async code in an
   apparently synchronous way.

You're making an interesting point to orient the conversation
though because it could orient the allocation of private data
in both design, if tasks are to be chained and share data.

It also raise a question of the use case we want to tackle
here. Preparser / thumbnailer use cases are likely to be an
interface / UI matter, which usually benefit from async, but
maybe there are other situation in which it would be suitable?

Maybe a summary of all points here would be useful too?

Regards,
--
Alexandre Janniaux
Videolabs