[vlc-devel] [RFC]: Reworking the metadata listener API => writting a proper filter API

[Steve Lhomme]

On 2021-03-15 9:52, Thomas Guillem wrote:
> The filter is inserted in the filter-chain of its type.
> 
> For audio-meter types, it's just before rendering, for classical audio-filter, it is after decoding.
> 
> On Mon, Mar 15, 2021, at 08:09, Steve Lhomme wrote:
>> It doesn't say when the filter is called. Is it before, during, after
>> rendering ?
>> In the case of (dynamic) HDR it should at least be just before or at the
>> same time the next frame will be displayed/sent. And probably not occur
>> if the frame is dropped if it's a state and not sent for each picture
>> (which should also be documented).
> 
> Then, you need to create a new filter type like I did for audio-meter, no ?

"just before rendering" is correct for dynamic HDR. But it's not a big 
issue when using rendering callbacks as the HDR data are already passed 
there at the right time.

>>
>> On 2021-03-05 16:48, Thomas Guillem wrote:
>>> Hello everyone,
>>>
>>> I recently added a generic metadata listener API into the new player.
>>> It's only used to get loudness measurements for now, cf.
>>> https://code.videolan.org/tguillem/vlc/-/commit/e8f17308146f446bc46f6a3b787aed2406ce8d31
>>> <https://code.videolan.org/tguillem/vlc/-/commit/e8f17308146f446bc46f6a3b787aed2406ce8d31>
>>> And I tried to port it to libvlc here:
>>> https://code.videolan.org/tguillem/vlc/-/commits/loudness-libvlc
>>> <https://code.videolan.org/tguillem/vlc/-/commits/loudness-libvlc>
>>>
>>> I received few valid critics with these 2 API (Core + LibVKC) from
>>> developers, and LibVLC users:
>>> - LibVLC users don't like using a new kind a listener API (that is not
>>> the classical event manager)
>>> - New kind of metadata will likely come from a filter.
>>> - Other kind of metadata (like HDR) can come from es_format_t. We need
>>> to find a proper way to forward es_format_t change to LibVLC in that case.
>>>
>>> So, maybe it is a good time to think about a proper filter API in LibVLC
>>> and Core. An API to manipulate filters of a given category (splitter,
>>> audio, meter, video, sub) from a specific output. This API will let
>>> users add, re-order, remove filters and also get notification from them
>>> (for the audio meter filter).
>>>
>>> Note that aout and vout are not (yet?) exposed to libvlc, so the libVLC
>>> will only be able to handle filters by category on the media_player.
>>>
>>> Here is my proposal for the Core API (in a form of a C header):
>>>
>>> typedef struct vlc_filter_entry_t vlc_filter_entry_t;
>>>
>>> /* All type of filters supported by VLC */
>>> enum vlc_filter_type
>>> {
>>>       VLC_PLAYER_FILTER_AUDIO,
>>>       /* Such filters are only useful if the user listen to events via
>>>        * vlc_filter_entry_NewAudioMeter() */
>>>       VLC_PLAYER_FILTER_AUDIO_METER,
>>>       VLC_PLAYER_FILTER_VIDEO,
>>>       VLC_PLAYER_FILTER_VIDEO_SPLITTER,
>>>       VLC_PLAYER_FILTER_SUBTITLE,
>>>       VLC_PLAYER_FILTER_SUBTITLE_SOURCE,
>>> };
>>>
>>> /* Listener, only used with audio_meter filters */
>>> struct vlc_filter_audio_meter_listener
>>> {
>>>       union {
>>>           /* Generic callback. For the loudness audio_meter, val1 will be a
>>>            * int64_t date, val2 will be a pointer to the vlc_audio_loudness
>>>            * struct */
>>>           void (*on_values_changed)(vlc_value_t val1, val_value_t val2);
>>>           /* Maybe add some other cb definition later depending on the
>>> need */
>>>       };
>>> };
>>>
>>> /* Create a new filter entry
>>> * The filter entry hold necessary informations to start a filter on the aout
>>> * or the vout.
>>> */
>>> VLC_API vlc_filter_entry_t *
>>> vlc_filter_entry_New(enum vlc_filter_type type, const char *name, const
>>> char *option);
>>>
>>> /* Create a new audio meter entry with a listener callback */
>>> VLC_API vlc_filter_entry_t *
>>> vlc_filter_entry_NewAudioMeter(const char *name,  const char *option,
>>> const struct vlc_filter_audio_meter_listener * cb, void *data);
>>>
>>> /* Return the name of the filter entry */
>>> VLC_API const char * vlc_filter_entry_GetName(vlc_filter_entry_t *fentry);
>>>
>>> /* Return the option of the filter entry */
>>> VLC_API const char * vlc_filter_entry_GetOption(vlc_filter_entry_t *fentry);
>>>
>>> /* Release a filter entry. Filter entries are refcounted internally (by
>>> the filter_list and the filter entry creator */
>>> VLC_API void vlc_filter_entry_Release(vlc_filter_entry_t *fentry);
>>>
>>> /* Filter entry list, owned by vout and aout. There is one filter list
>>> per filter type */
>>>
>>> typedef struct vlc_filter_list_t vlc_filter_list;
>>>
>>> /* Lock the filter list, read and write operations need to be done locked */
>>> void vlc_filter_list_Lock(vlc_filter_list_t *list);
>>>
>>> /* Unlock the filter list, and commit the written changes if needed. To
>>> commit
>>> * any changes, the list will change the corresponding filter option (like
>>> * "video-splitter") on the corresponding vout/aout. */
>>> void vlc_filter_list_Unlock(vlc_filter_list_t *list);
>>>
>>> /* Return the number of filters */
>>> size_t vlc_filter_list_GetCount(vlc_filter_list_t *list);
>>>
>>> /* Get the filter entry at the specific index */
>>> vlc_filter_entry_t * vlc_filter_list_GetFilterAt(vlc_filter_list_t
>>> *list, size_t index);
>>>
>>> /* Add a new filter at a specific position or move a filter at the new
>>> * position. */
>>> void
>>> vlc_filter_list_MoveFilterAt(vlc_filter_list_t *list, size_t index,
>>> vlc_filter_entry_t *filter);
>>>
>>> /* Append a filter at the end of the list */
>>> static inline void
>>> vlc_filter_list_AppendFilter(vlc_filter_list_t *list, vlc_filter_entry_t
>>> *filter)
>>> {
>>>       vlc_filter_list_MoveFilterAt(list, vlc_filter_list_GetCount(list),
>>> filter);
>>> }
>>>
>>> /* Remove a filter from the list. */
>>> void vlc_filter_list_RemoveFilter(vlc_filter_list_t *list,
>>> vlc_filter_entry_t *filter);
>>>
>>> /* Get the filter list of the aout for the specific type */
>>> vlc_filter_list_t *
>>> aout_GetFilterList(aout_output_t *, enum vlc_filter_type type);
>>>
>>> /* Get the filter list of the vout for the specific type */
>>> vlc_filter_list_t *
>>> vout_GetFilterList(vout_output_t *, enum vlc_filter_type type);
>>>
>>> Comments are more than welcome!
>>>
>>> Best,
>>> Thomas
>>>
>>> _______________________________________________
>>> vlc-devel mailing list
>>> To unsubscribe or modify your subscription options:
>>> https://mailman.videolan.org/listinfo/vlc-devel
>>>
>> _______________________________________________
>> vlc-devel mailing list
>> To unsubscribe or modify your subscription options:
>> https://mailman.videolan.org/listinfo/vlc-devel
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
>