[vlc-devel] [RFC]: Reworking the metadata listener API => writting a proper filter API

[Thomas Guillem]

The filter is inserted in the filter-chain of its type.

For audio-meter types, it's just before rendering, for classical audio-filter, it is after decoding.

On Mon, Mar 15, 2021, at 08:09, Steve Lhomme wrote:
> It doesn't say when the filter is called. Is it before, during, after 
> rendering ?
> In the case of (dynamic) HDR it should at least be just before or at the 
> same time the next frame will be displayed/sent. And probably not occur 
> if the frame is dropped if it's a state and not sent for each picture 
> (which should also be documented).

Then, you need to create a new filter type like I did for audio-meter, no ?

> 
> On 2021-03-05 16:48, Thomas Guillem wrote:
> > Hello everyone,
> > 
> > I recently added a generic metadata listener API into the new player. 
> > It's only used to get loudness measurements for now, cf. 
> > https://code.videolan.org/tguillem/vlc/-/commit/e8f17308146f446bc46f6a3b787aed2406ce8d31 
> > <https://code.videolan.org/tguillem/vlc/-/commit/e8f17308146f446bc46f6a3b787aed2406ce8d31>
> > And I tried to port it to libvlc here: 
> > https://code.videolan.org/tguillem/vlc/-/commits/loudness-libvlc 
> > <https://code.videolan.org/tguillem/vlc/-/commits/loudness-libvlc>
> > 
> > I received few valid critics with these 2 API (Core + LibVKC) from 
> > developers, and LibVLC users:
> > - LibVLC users don't like using a new kind a listener API (that is not 
> > the classical event manager)
> > - New kind of metadata will likely come from a filter.
> > - Other kind of metadata (like HDR) can come from es_format_t. We need 
> > to find a proper way to forward es_format_t change to LibVLC in that case.
> > 
> > So, maybe it is a good time to think about a proper filter API in LibVLC 
> > and Core. An API to manipulate filters of a given category (splitter, 
> > audio, meter, video, sub) from a specific output. This API will let 
> > users add, re-order, remove filters and also get notification from them 
> > (for the audio meter filter).
> > 
> > Note that aout and vout are not (yet?) exposed to libvlc, so the libVLC 
> > will only be able to handle filters by category on the media_player.
> > 
> > Here is my proposal for the Core API (in a form of a C header):
> > 
> > typedef struct vlc_filter_entry_t vlc_filter_entry_t;
> > 
> > /* All type of filters supported by VLC */
> > enum vlc_filter_type
> > {
> >      VLC_PLAYER_FILTER_AUDIO,
> >      /* Such filters are only useful if the user listen to events via
> >       * vlc_filter_entry_NewAudioMeter() */
> >      VLC_PLAYER_FILTER_AUDIO_METER,
> >      VLC_PLAYER_FILTER_VIDEO,
> >      VLC_PLAYER_FILTER_VIDEO_SPLITTER,
> >      VLC_PLAYER_FILTER_SUBTITLE,
> >      VLC_PLAYER_FILTER_SUBTITLE_SOURCE,
> > };
> > 
> > /* Listener, only used with audio_meter filters */
> > struct vlc_filter_audio_meter_listener
> > {
> >      union {
> >          /* Generic callback. For the loudness audio_meter, val1 will be a
> >           * int64_t date, val2 will be a pointer to the vlc_audio_loudness
> >           * struct */
> >          void (*on_values_changed)(vlc_value_t val1, val_value_t val2);
> >          /* Maybe add some other cb definition later depending on the 
> > need */
> >      };
> > };
> > 
> > /* Create a new filter entry
> > * The filter entry hold necessary informations to start a filter on the aout
> > * or the vout.
> > */
> > VLC_API vlc_filter_entry_t *
> > vlc_filter_entry_New(enum vlc_filter_type type, const char *name, const 
> > char *option);
> > 
> > /* Create a new audio meter entry with a listener callback */
> > VLC_API vlc_filter_entry_t *
> > vlc_filter_entry_NewAudioMeter(const char *name,  const char *option, 
> > const struct vlc_filter_audio_meter_listener * cb, void *data);
> > 
> > /* Return the name of the filter entry */
> > VLC_API const char * vlc_filter_entry_GetName(vlc_filter_entry_t *fentry);
> > 
> > /* Return the option of the filter entry */
> > VLC_API const char * vlc_filter_entry_GetOption(vlc_filter_entry_t *fentry);
> > 
> > /* Release a filter entry. Filter entries are refcounted internally (by 
> > the filter_list and the filter entry creator */
> > VLC_API void vlc_filter_entry_Release(vlc_filter_entry_t *fentry);
> > 
> > /* Filter entry list, owned by vout and aout. There is one filter list 
> > per filter type */
> > 
> > typedef struct vlc_filter_list_t vlc_filter_list;
> > 
> > /* Lock the filter list, read and write operations need to be done locked */
> > void vlc_filter_list_Lock(vlc_filter_list_t *list);
> > 
> > /* Unlock the filter list, and commit the written changes if needed. To 
> > commit
> > * any changes, the list will change the corresponding filter option (like
> > * "video-splitter") on the corresponding vout/aout. */
> > void vlc_filter_list_Unlock(vlc_filter_list_t *list);
> > 
> > /* Return the number of filters */
> > size_t vlc_filter_list_GetCount(vlc_filter_list_t *list);
> > 
> > /* Get the filter entry at the specific index */
> > vlc_filter_entry_t * vlc_filter_list_GetFilterAt(vlc_filter_list_t 
> > *list, size_t index);
> > 
> > /* Add a new filter at a specific position or move a filter at the new
> > * position. */
> > void
> > vlc_filter_list_MoveFilterAt(vlc_filter_list_t *list, size_t index, 
> > vlc_filter_entry_t *filter);
> > 
> > /* Append a filter at the end of the list */
> > static inline void
> > vlc_filter_list_AppendFilter(vlc_filter_list_t *list, vlc_filter_entry_t 
> > *filter)
> > {
> >      vlc_filter_list_MoveFilterAt(list, vlc_filter_list_GetCount(list), 
> > filter);
> > }
> > 
> > /* Remove a filter from the list. */
> > void vlc_filter_list_RemoveFilter(vlc_filter_list_t *list, 
> > vlc_filter_entry_t *filter);
> > 
> > /* Get the filter list of the aout for the specific type */
> > vlc_filter_list_t *
> > aout_GetFilterList(aout_output_t *, enum vlc_filter_type type);
> > 
> > /* Get the filter list of the vout for the specific type */
> > vlc_filter_list_t *
> > vout_GetFilterList(vout_output_t *, enum vlc_filter_type type);
> > 
> > Comments are more than welcome!
> > 
> > Best,
> > Thomas
> > 
> > _______________________________________________
> > vlc-devel mailing list
> > To unsubscribe or modify your subscription options:
> > https://mailman.videolan.org/listinfo/vlc-devel
> > 
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel