[vlc-devel] [RFC 0/2] New executor API

[Romain Vimont]

The background worker API works, but it turns out to be complex to use (IMO).

Everything is void*, and it's difficult to know intuitively what is each
"thing". For example:
 - int (*pf_start)(void *owner, void *entity, void **out)
 - int (*pf_probe)(void *owner, void *handle)

In addition, the user must provide callbacks to hold/release "entities".

I would like to propose a new API, hopefully easier to use.

 - Patch 1 introduces the proposed API and its implementation.
 - Patch 2 uses it in the preparser to replace the background worker.

Basically, submitting a new task from an executor thread consists in calling
vlc_executor_Submit() with a "runnable" (a structure with a run() function).

In order to support the background worker features, a timeout can be provided
to automatically cancel/interrupt the task (which adds quite a lot of
complexity).

The major behavior difference between background worker and executor comes from
how they execute tasks.

For each task, a _background worker_ thread executes the following steps:
 - compute the task deadline (from its timeout)
 - execute the start() callback, assumed to quickly start an asynchronous task
   (e.g. in a separate thread, handled by the user)
 - wait until the deadline (or until the task is complete, signaled by a
   "probe" mechanism)
 - execute the stop() callback

By contrast, for each task, an _executor_ thread executes the following steps:
 - if the task requests a timeout, make sure that interruption is managed
 - execute the task synchronously

Since the executor threads are now used to execute the runnable synchronously,
an additional thread (no more than 1 per executor) is used to "interrupt"
tasks.

The logic is different, so the complexity does not appear in the same place in
the same cases. My hope is that this API is more intuitive.

The API and naming is inpired from ExecutorService in Java (but way more
limited):
<https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/concurrent/ExecutorService.html>

>From background worker, I kept the principle to pass a user-provided void* id
parameter, used to cancel tasks, because it is directly used by public APIs,
like libvlc_MetadataRequest().

(In theory, I would probably have preferred to return some vlc_future_t that
would be cancelable, but it would also add complexity, so ignore it for now.)

Here is a branch: https://code.videolan.org/rom1v/vlc/commits/executor

Thank you for your feedbacks.

Regards,
Romain

Romain Vimont (2):
  executor: introduce new executor API
  preparser: use the new executor API

 include/vlc_executor.h    | 194 ++++++++++++++
 src/Makefile.am           |   1 +
 src/libvlccore.sym        |   4 +
 src/misc/executor.c       | 545 ++++++++++++++++++++++++++++++++++++++
 src/preparser/preparser.c | 342 +++++++++++++-----------
 5 files changed, 931 insertions(+), 155 deletions(-)
 create mode 100644 include/vlc_executor.h
 create mode 100644 src/misc/executor.c

-- 
2.28.0