[vlc-devel] [RFC 0/2] New executor API

Alexandre Janniaux ajanni at videolabs.io
Tue Sep 1 11:00:07 CEST 2020


Hi,

On Tue, Sep 01, 2020 at 08:21:16AM +0200, Steve Lhomme wrote:
> On 2020-08-31 19:43, Pierre Ynard via vlc-devel wrote:
> >
> > Subject:
> > Re: [vlc-devel] [RFC 0/2] New executor API
> > From:
> > Pierre Ynard <linkfanel at yahoo.fr>
> > Date:
> > 2020-08-31, 19:43
> > To:
> > vlc-devel at videolan.org
> >
> > To:
> > vlc-devel at videolan.org
> >
> >
> > > I've never seen a scheduler where task owners just forgot their tasks
> > > and let the scheduler track the task on behalf of the owners. That
> > > misses separation of concerns. Besides I can't see how it works - the
> > > owner needs to expect the result of their queued tasks. For instance,
> > > if they're futures, it needs a reference to those futures.
> >
> > I have. The tasks were chained. Apparently it worked, much better than
> > the previous code.
>
> Not only I have, too. But that's the only way I've ever used. But that was
> in Java which has a garbage collector, so a different management of
> resources. Also I never had to care about canceling tasks. The only use case
> to cancel pending jobs was just before app shutdown, and you can always
> shutdown the whole executor and it will take care of pending tasks.

Yes, that's because java has exception and can trigger them
in sync with the code being executed (see InterruptedException).
When it exits, it has three choices:

 - wait until all tasks are completed
 - trigger an exception in all task and wait until all tasks are
   completed
 - remove idle tasks, trigger an exception, wait untile all tasks
   are completed

It can do the second and third points easily because of this
exception mechanism and because everything is allocated on the
heap, and deallocated through a garbage collector.

That's an important point to take into account for the design
of such executor in C.

> I think the preparser is a good candidate of "launch tasks, I don't care
> what happens to them".

I strongly disagree to your sentence as is. Preparse results
need to be forwarded, probably to an interface, otherwise
they are useless.

In the current implementation, I also disagree with you
becaues the preparser does care about the tasks, since
it might cancel them.

What I write here doesn't define which design is best though
or provide more depth to this conversation.

> Some Medialibrary tasks like "rename a file and update the database after"
> is another example of chaining tasks which the caller doesn't care about
> what happens in the end.

That's actually a good example of task for the current
conversation. How would you implement the cancellation of
the task in the case the renaming has been done but not the
update of the database?

In practice you can probably just say that you do the two
tasks in only one task and handle the cancellation before or
after the execution of the two tasks but not in between, but
that's a choice which need to be taken into account by the
executor. Saying «you need to handle the cancellation, I'll
just wait for the task to finish» is still much simpler
since there is no choice to make here, only usual design that
we already have in access, fingerprinter, etc.

> There still needs to be some resource management like file/database/socket
> locking, but that doesn't have to be in the owner (in can but the design
> should not force it).

What do you call owner here?

Regards,
--
Alexandre Janniaux
Videolabs


More information about the vlc-devel mailing list