[vlc-devel] [PATCH v2 3/7] aout: add gapless support
Thomas Guillem
thomas at gllm.fr
Thu Mar 21 14:34:04 CET 2019
On Thu, Mar 21, 2019, at 14:29, Rémi Denis-Courmont wrote:
> No. You are making unwarranted assumptions, and not just about no audio ES. I cannot agree.
That is why I will start the next input, and pause it before it request new ouputs (vout/aout/spu/aout).
Then, if there is an audio track, I'll compare the format to the current one and enable gapless accordingly.
Why unwarranted assumptions am I making in that case ?
>
> Le 21 mars 2019 20:49:49 GMT+08:00, Thomas Guillem <thomas at gllm.fr> a écrit :
>>
>> On Thu, Mar 21, 2019, at 13:42, Rémi Denis-Courmont wrote:
>>> I already told you that this cannot work. If you think about all possible combos, it's kinda obvious that concurrent inputs are necessary, but not sufficient, to make it work.
>>
>> Not obvious at all. It works for most common case: playing local or network music album. The buffer of the audio output is generally around 500ms before a drain, this leave a *lot* of time to open the next input. I can even play a music cut into several 200ms pieces without any problem.
>> And for the case where the audio buffer size is not enough ? It's handled: gapless will be canceled (drain/flush last stream and open the new one).
>>
>>>
>>> You cannot know if PTS will be continuous or if the formats will match, or if there will be an active audio ES at all.
>>
>> My current branch handle the case when PTS are not continuous or if format won't match.
>>
>> Only your last point is not handled. This last point need to have concurrent input: activate gapless only if the next media is sure to have an active audio ES.
>>
>>>
>>> Le 21 mars 2019 01:09:10 GMT+08:00, Thomas Guillem <thomas at gllm.fr> a écrit :
>>>> The player will call aout_OutputSetGaplessEnabled(true|false) when there is a
>>>> next media to play or not. This call is protected with the same mutex used for
>>>> start/stop and can be called from anywhere.
>>>>
>>>> Then, when the stream is draining (aout_DecDrain), it will postpone the actual
>>>> drain if gapless is enabled, and do the following depending on the next aout
>>>> call:
>>>>
>>>> - if play is called just after, the postponed drain is executed and the
>>>> gapless state is canceled.
>>>> - if flush is called, the gapless state is canceled and everything is flushed
>>>> - if stop is called, filters are cleaned but the stream is not stopped, it is
>>>> saved for a future usage: (via aout_OutputSaveStream()).
>>>>
>>>> On the next stream start, it will try to recover the previous stream (that
>>>> could be stopped if the player disabled gapless in the meantime) and will do
>>>> the following depending on the aout_DecNew parameters:
>>>>
>>>> - if format are different: cancel gapless state, drain, flush and stop the
>>>> previous stream.
>>>>
>>>> - if format are equals: use the previous stream (don't call aout->stop(),
>>>> aout->start()), and set-up new filters chain with the new clock.
>>>>
>>>> The final gapless transition will be done from aout_DecPlay():
>>>>
>>>> - if the previous stream delay (time_get) is compatible with the next clock,
>>>> blocks will be added in a temporary buffer until the clock jitter delay is
>>>> reached.
>>>> - if not, cancel the gapless state, drain and flush the (previous=current)
>>>> stream (but don't restart it), and play all blocks normally.
>>>>
>>>> When the jitter delay is reached, blocks added in the temporary buffers can be
>>>> played and the playback can resume normally.
>>>>
>>>> Fixes #549 src/audio_output/aout_internal.h | 17 +++
>>>> src/audio_output/dec.c | 206 +++++++++++++++++++++++++++++--
>>>> src/audio_output/output.c | 74 +++++++++++
>>>> 3 files changed, 289 insertions(+), 8 deletions(-)
>>>>
>>>> diff --git a/src/audio_output/aout_internal.h b/src/audio_output/aout_internal.h
>>>> index 12fa4dfad9..bab59977eb 100644
>>>> --- a/src/audio_output/aout_internal.h
>>>> +++ b/src/audio_output/aout_internal.h
>>>> @@ -47,6 +47,8 @@ typedef struct
>>>> aout_filters_t *filters;
>>>> aout_volume_t *volume;
>>>>
>>>> + bool stream_saved;
>>>> +
>>>> struct
>>>> {
>>>> vlc_mutex_t lock;
>>>> @@ -85,6 +87,17 @@ typedef struct
>>>> atomic_uint buffers_played;
>>>> atomic_uchar restart;
>>>>
>>>> + bool gapless_enabled;
>>>> + enum
>>>> + {
>>>> + GAPLESS_STATE_NONE,
>>>> + GAPLESS_STATE_DRAINED,
>>>> + GAPLESS_STATE_STOPPED,
>>>> + GAPLESS_STATE_STARTED,
>>>> + } gapless_state;
>>>> + block_t *gapless_block_chain;
>>>> + block_t **gapless_block_last;
>>>> +
>>>> atomic_uintptr_t refs;
>>>> } aout_owner_t;
>>>>
>>>> @@ -120,6 +133,10 @@ void aout_Destroy (audio_output_t *);
>>>> int aout_OutputNew(audio_output_t *, audio_sample_format_t *,
>>>> aout_filters_cfg_t *filters_cfg);
>>>> void aout_OutputDelete( audio_output_t * p_aout );
>>>> +void aout_OutputSetGaplessEnabled(audio_output_t *aout, bool enabled);
>>>> +bool aout_OutputIsGaplessEnabled(audio_output_t *aout);
>>>> +int aout_OutputRecoverStream(audio_output_t *aout);
>>>> +void aout_OutputSaveStream(audio_output_t *aout);
>>>>
>>>>
>>>> /* From common.c : */
>>>> diff --git a/src/audio_output/dec.c b/src/audio_output/dec.c
>>>> index 75ea4e6621..e4f1737776 100644
>>>> --- a/src/audio_output/dec.c
>>>> +++ b/src/audio_output/dec.c
>>>> @@ -65,6 +65,44 @@ static bool aout_FormatEquals(const audio_sample_format_t *f1,
>>>> && f1->i_channels == f2->i_channels;
>>>> }
>>>>
>>>> +static int aout_GaplessReuseStream(audio_output_t *aout,
>>>> + const audio_sample_format_t *format,
>>>> + vlc_clock_t *clock,
>>>> + const audio_replay_gain_t *replay_gain)
>>>> +{
>>>> + aout_owner_t *owner = aout_owner(aout);
>>>> +
>>>> + if (aout_OutputRecoverStream(aout) != VLC_SUCCESS)
>>>> + {
>>>> + /* Unlikely case: The old stream was already stopped by the player */
>>>> + owner->gapless_state = GAPLESS_STATE_NONE;
>>>> + return -1;
>>>> + }
>>>> +
>>>> + if (!aout_FormatEquals(format, &owner->input_format))
>>>> + goto fail;
>>>> +
>>>> + /* Reuse the previous stream but set-up new clock and filters */
>>>> + owner->sync.clock = clock;
>>>> + owner->filters =
>>>> + aout_FiltersNewWithClock(VLC_OBJECT(aout), clock,
>>>> + format, &owner->mixer_format,
>>>> + &owner->filters_cfg);
>>>> + if (owner->filters == NULL)
>>>> + goto fail;
>>>> +
>>>> + owner->volume = aout_volume_New (aout, replay_gain);
>>>> + owner->gapless_state = GAPLESS_STATE_STARTED;
>>>> + msg_Dbg(aout, "trying gapless transition");
>>>> + return 0;
>>>> +fail:
>>>> + /* No gapless: drain and delete the old stream */
>>>> + aout_Drain(aout);
>>>> + aout_OutputDelete(aout);
>>>> + owner->gapless_state = GAPLESS_STATE_NONE;
>>>> + return -1;
>>>> +}
>>>> +
>>>> /**
>>>> * Creates an audio output
>>>> */
>>>> @@ -74,6 +112,16 @@ int aout_DecNew(audio_output_t *p_aout, const audio_sample_format_t *p_format,
>>>> assert(p_aout);
>>>> assert(p_format);
>>>> assert(clock);
>>>> +
>>>> + aout_owner_t *owner = aout_owner(p_aout);
>>>> +
>>>> + assert(owner->gapless_state == GAPLESS_STATE_NONE
>>>> + || owner->gapless_state == GAPLESS_STATE_STOPPED);
>>>> +
>>>> + if (owner->gapless_state == GAPLESS_STATE_STOPPED
>>>> + && aout_GaplessReuseStream(p_aout, p_format, clock, p_replay_gain) == 0)
>>>> + return 0;
>>>> +
>>>> if( p_format->i_bitspersample > 0 )
>>>> {
>>>> /* Sanitize audio format, input need to have a valid physical channels
>>>> @@ -100,8 +148,6 @@ int aout_DecNew(audio_output_t *p_aout, const audio_sample_format_t *p_format,
>>>> return -1;
>>>> }
>>>>
>>>> - aout_owner_t *owner = aout_owner(p_aout);
>>>> -
>>>> /* Create the audio output stream */
>>>> owner->volume = aout_volume_New (p_aout, p_replay_gain);
>>>>
>>>> @@ -147,11 +193,25 @@ void aout_DecDelete (audio_output_t *aout)
>>>> {
>>>> aout_owner_t *owner = aout_owner (aout);
>>>>
>>>> + assert(owner->gapless_state == GAPLESS_STATE_NONE
>>>> + || owner->gapless_state == GAPLESS_STATE_DRAINED);
>>>> +
>>>> if (owner->mixer_format.i_format)
>>>> {
>>>> - aout_DecFlush(aout);
>>>> - aout_FiltersDelete (aout, owner->filters);
>>>> - aout_OutputDelete (aout);
>>>> + if (owner->gapless_state == GAPLESS_STATE_DRAINED)
>>>> + {
>>>> + /* Gapless transition: save the stream for the next aout_DecNew
>>>> + * call */
>>>> + aout_OutputSaveStream(aout);
>>>> + owner->gapless_state = GAPLESS_STATE_STOPPED;
>>>> + aout_FiltersDelete (aout, owner->filters);
>>>> + }
>>>> + else
>>>> + {
>>>> + aout_DecFlush(aout);
>>>> + aout_FiltersDelete (aout, owner->filters);
>>>> + aout_OutputDelete (aout);
>>>> + }
>>>> }
>>>> aout_volume_Delete (owner->volume);
>>>> owner->volume = NULL;
>>>> @@ -206,6 +266,7 @@ static int aout_CheckReady (audio_output_t *aout)
>>>> }
>>>> aout_FiltersSetClockDelay(owner->filters, owner->sync.delay);
>>>> }
>>>> + owner->gapless_state = GAPLESS_STATE_NONE;
>>>> /* TODO: This would be a good time to call clean up any video output
>>>> * left over by an audio visualization:
>>>> input_resource_TerminatVout(MAGIC HERE); */
>>>> @@ -426,6 +487,89 @@ static void aout_Play(audio_output_t *aout, block_t *block,
>>>> atomic_fetch_add_explicit(&owner->buffers_played, 1, memory_order_relaxed);
>>>> }
>>>>
>>>> +static void aout_GaplessPlay(audio_output_t *aout, block_t *block,
>>>> + vlc_tick_t system_now, vlc_tick_t original_pts)
>>>> +{
>>>> + aout_owner_t *owner = aout_owner (aout);
>>>> + vlc_tick_t delay;
>>>> +
>>>> + assert(owner->gapless_state == GAPLESS_STATE_STARTED);
>>>> +
>>>> + /* Save blocks until we reach the new clock jitter delay or until we cancel
>>>> + * gapless due to timing issues */
>>>> + if (block != NULL)
>>>> + {
>>>> + block->i_dts = original_pts;
>>>> + block_ChainLastAppend(&owner->gapless_block_last, block);
>>>> + }
>>>> +
>>>> + if (aout->time_get(aout, &delay) != 0)
>>>> + {
>>>> + msg_Warn(aout, "aout delay invalid: canceling gapless");
>>>> + goto cancel;
>>>> + }
>>>> +
>>>> + const vlc_tick_t play_date =
>>>> + vlc_clock_ConvertToSystem(owner->sync.clock, system_now, 0,
>>>> + owner->sync.rate);
>>>> + const vlc_tick_t jitter = play_date - system_now;
>>>> +
>>>> + if (delay <= jitter)
>>>> + {
>>>> + msg_Warn(aout, "clock jitter higher than aout delay: canceling gapless");
>>>> + goto cancel;
>>>> + }
>>>> +
>>>> + if (jitter >= 0 && block != NULL)
>>>> + return; /* need more blocks */
>>>> +
>>>> + if (delay <= AOUT_MAX_PTS_ADVANCE)
>>>> + {
>>>> + msg_Warn(aout, "delay too short: canceling gapless");
>>>> + goto cancel;
>>>> + }
>>>> +
>>>> + if (owner->gapless_block_chain)
>>>> + {
>>>> + const vlc_tick_t block_original_pts = owner->gapless_block_chain->i_dts;
>>>> + const vlc_tick_t drift =
>>>> + vlc_clock_Update(owner->sync.clock, system_now + delay,
>>>> + block_original_pts, owner->sync.rate);
>>>> + if (drift != 0)
>>>> + {
>>>> + msg_Warn(aout, "can't update clock: canceling gapless");
>>>> + goto cancel;
>>>> + }
>>>> + }
>>>> +
>>>> + msg_Dbg(aout, "gapless transition successful");
>>>> + owner->gapless_state = GAPLESS_STATE_NONE;
>>>> + owner->sync.discontinuity = false;
>>>> +
>>>> +cancel:
>>>> + if (owner->gapless_state != GAPLESS_STATE_NONE)
>>>> + {
>>>> + /* Gapless canceled: drain and flush the previous stream */
>>>> + aout_Drain(aout);
>>>> + aout_DecFlush(aout);
>>>> + owner->gapless_state = GAPLESS_STATE_NONE;
>>>> + }
>>>> +
>>>> + /* Play every saved blocks now */
>>>> + for (block_t *next = NULL, *it = owner->gapless_block_chain; it; it = next)
>>>> + {
>>>> + next = it->p_next;
>>>> + const vlc_tick_t block_original_pts = it->i_dts;
>>>> + it->i_dts = VLC_TICK_INVALID;
>>>> +
>>>> + aout_Play(aout, it, system_now, block_original_pts);
>>>> + system_now = vlc_tick_now(); /* system_now might be too old */
>>>> + }
>>>> +
>>>> + owner->gapless_block_chain = NULL;
>>>> + owner->gapless_block_last = &owner->gapless_block_chain;
>>>> +}
>>>> +
>>>> /*****************************************************************************
>>>> * aout_DecPlay : filter & mix the decoded buffer
>>>> *****************************************************************************/
>>>> @@ -433,6 +577,17 @@ int aout_DecPlay(audio_output_t *aout, block_t *block)
>>>> {
>>>> aout_owner_t *owner = aout_owner (aout);
>>>>
>>>> + if (unlikely(owner->gapless_state == GAPLESS_STATE_DRAINED))
>>>> + {
>>>> + /* Drained but not stopped: do the drain that was previously ignored
>>>> + * and cancel the gapless transition */
>>>> + aout_Drain(aout);
>>>> + owner->gapless_state = GAPLESS_STATE_NONE;
>>>> + }
>>>> +
>>>> + assert(owner->gapless_state == GAPLESS_STATE_NONE
>>>> + || owner->gapless_state == GAPLESS_STATE_STARTED);
>>>> +
>>>> assert (block->i_pts != VLC_TICK_INVALID);
>>>>
>>>> block->i_length = vlc_tick_from_samples( block->i_nb_samples,
>>>> @@ -476,7 +631,8 @@ int aout_DecPlay(audio_output_t *aout, block_t *block)
>>>> aout_volume_Amplify (owner->volume, block);
>>>>
>>>> /* Update delay */
>>>> - if (owner->sync.request_delay != owner->sync.delay)
>>>> + if (owner->gapless_state == GAPLESS_STATE_NONE
>>>> + && owner->sync.request_delay != owner->sync.delay)
>>>> {
>>>> owner->sync.delay = owner->sync.request_delay;
>>>> vlc_tick_t delta = vlc_clock_SetDelay(owner->sync.clock, owner->sync.delay);
>>>> @@ -486,7 +642,12 @@ int aout_DecPlay(audio_output_t *aout, block_t *block)
>>>> }
>>>>
>>>> vlc_tick_t system_now = vlc_tick_now();
>>>> - aout_Play(aout, block, system_now, original_pts);
>>>> +
>>>> + if (owner->gapless_state == GAPLESS_STATE_STARTED)
>>>> + aout_GaplessPlay(aout, block, system_now, original_pts);
>>>> + else
>>>> + aout_Play(aout, block, system_now, original_pts);
>>>> +
>>>> return ret;
>>>> drop:
>>>> owner->sync.discontinuity = true;
>>>> @@ -540,6 +701,15 @@ void aout_DecFlush(audio_output_t *aout)
>>>>
>>>> if (owner->mixer_format.i_format)
>>>> {
>>>> + if (owner->gapless_state != GAPLESS_STATE_NONE)
>>>> + {
>>>> + /* Flush gapless buffers and reset state */
>>>> + block_ChainRelease(owner->gapless_block_chain);
>>>> + owner->gapless_block_chain = NULL;
>>>> + owner->gapless_block_last = &owner->gapless_block_chain;
>>>> + owner->gapless_state = GAPLESS_STATE_NONE;
>>>> + }
>>>> +
>>>> aout_FiltersFlush (owner->filters);
>>>>
>>>> aout->flush(aout);
>>>> @@ -571,11 +741,31 @@ void aout_DecDrain(audio_output_t *aout)
>>>> if (!owner->mixer_format.i_format)
>>>> return;
>>>>
>>>> + assert(owner->gapless_state == GAPLESS_STATE_NONE
>>>> + || owner->gapless_state == GAPLESS_STATE_STARTED);
>>>> +
>>>> + if (owner->gapless_state == GAPLESS_STATE_STARTED)
>>>> + {
>>>> + /* Drain gapless block chain */
>>>> + aout_GaplessPlay(aout, NULL, vlc_tick_now(), VLC_TICK_INVALID);
>>>> + }
>>>> +
>>>> block_t *block = aout_FiltersDrain (owner->filters);
>>>> if (block)
>>>> aout->play(aout, block, vlc_tick_now());
>>>>
>>>> - aout_Drain(aout);
>>>> + if (AOUT_FMT_LINEAR(&owner->mixer_format)
>>>> + && aout_OutputIsGaplessEnabled(aout))
>>>> + {
>>>> +
>>>> + /* First gapless state, cancellable if stop/play/flush is not called
>>>> + * right after */
>>>> + owner->gapless_state = GAPLESS_STATE_DRAINED;
>>>> + owner->gapless_block_chain = NULL;
>>>> + owner->gapless_block_last = &owner->gapless_block_chain;
>>>> + }
>>>> + else
>>>> + aout_Drain(aout);
>>>>
>>>> vlc_clock_Reset(owner->sync.clock);
>>>> aout_FiltersResetClock(owner->filters);
>>>> diff --git a/src/audio_output/output.c b/src/audio_output/output.c
>>>> index 5c378dbb31..6f11628a52 100644
>>>> --- a/src/audio_output/output.c
>>>> +++ b/src/audio_output/output.c
>>>> @@ -223,6 +223,9 @@ audio_output_t *aout_New (vlc_object_t *parent)
>>>> vlc_viewpoint_init (&owner->vp.value);
>>>> atomic_init (&owner->vp.update, false);
>>>> atomic_init(&owner->refs, 0);
>>>> + owner->stream_saved = false;
>>>> + owner->gapless_enabled = false;
>>>> + owner->gapless_state = GAPLESS_STATE_NONE;
>>>>
>>>> /* Audio output module callbacks */
>>>> var_Create (aout, "volume", VLC_VAR_FLOAT);
>>>> @@ -565,6 +568,7 @@ int aout_OutputNew (audio_output_t *aout, audio_sample_format_t *restrict fmt,
>>>> aout->current_sink_info.headphones = false;
>>>>
>>>> vlc_mutex_lock(&owner->lock);
>>>> + assert(!owner->stream_saved);
>>>> int ret = aout->start(aout, fmt);
>>>> assert(aout->flush && aout->play && aout->time_get && aout->pause);
>>>> vlc_mutex_unlock(&owner->lock);
>>>> @@ -592,10 +596,80 @@ void aout_OutputDelete (audio_output_t *aout)
>>>> {
>>>> aout_owner_t *owner = aout_owner(aout);
>>>> vlc_mutex_lock(&owner->lock);
>>>> + assert(!owner->stream_saved);
>>>> aout->stop (aout);
>>>> vlc_mutex_unlock(&owner->lock);
>>>> }
>>>>
>>>> +/**
>>>> + * Recover the ownership of the stream previously saved
>>>> + */
>>>> +int aout_OutputRecoverStream(audio_output_t *aout)
>>>> +{
>>>> + aout_owner_t *owner = aout_owner(aout);
>>>> + vlc_mutex_lock(&owner->lock);
>>>> + if (unlikely(!owner->stream_saved))
>>>> + {
>>>> + /* The stream was already stopped by a previous
>>>> + * aout_OutputSetGaplessEnabled(false) from an other thread */
>>>> + vlc_mutex_unlock(&owner->lock);
>>>> + return VLC_EGENERIC;
>>>> + }
>>>> +
>>>> + owner->stream_saved = false;
>>>> + vlc_mutex_unlock(&owner->lock);
>>>> +
>>>> + return VLC_SUCCESS;
>>>> +}
>>>> +
>>>> +/**
>>>> + * Save the stream for a future usage
>>>> + */
>>>> +void aout_OutputSaveStream(audio_output_t *aout)
>>>> +{
>>>> + aout_owner_t *owner = aout_owner(aout);
>>>> +
>>>> + vlc_mutex_lock(&owner->lock);
>>>> +
>>>> + assert(!owner->stream_saved);
>>>> +
>>>> + /* The caller of aout_OutputSetGaplessEnabled (the player) is now the owner
>>>> + * of the stream during gapless transition. */
>>>> + owner->stream_saved = true;
>>>> + vlc_mutex_unlock(&owner->lock);
>>>> +}
>>>> +
>>>> +/**
>>>> + * Enable or disable gapless
>>>> + *
>>>> + * Can be called by the player when the next media changes
>>>> + */
>>>> +void aout_OutputSetGaplessEnabled(audio_output_t *aout, bool enabled)
>>>> +{
>>>> + aout_owner_t *owner = aout_owner (aout);
>>>> +
>>>> + vlc_mutex_lock(&owner->lock);
>>>> +
>>>> + owner->gapless_enabled = enabled;
>>>> +
>>>> + if (!enabled && unlikely(owner->stream_saved))
>>>> + {
>>>> + aout->stop (aout);
>>>> + owner->stream_saved = false;
>>>> + }
>>>> +
>>>> + vlc_mutex_unlock(&owner->lock);
>>>> +}
>>>> +
>>>> +bool aout_OutputIsGaplessEnabled(audio_output_t *aout)
>>>> +{
>>>> + aout_owner_t *owner = aout_owner(aout);
>>>> + vlc_mutex_lock(&owner->lock);
>>>> + const bool enabled = owner->gapless_enabled;
>>>> + vlc_mutex_unlock(&owner->lock);
>>>> + return enabled;
>>>> +}
>>>> +
>>>> /**
>>>> * Gets the volume of the audio output stream (independent of mute).
>>>> * \return Current audio volume (0. = silent, 1. = nominal),
>>>
>>> --
>>> Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma brièveté.
>>> _______________________________________________
>>> vlc-devel mailing list
>>> To unsubscribe or modify your subscription options:
>>> https://mailman.videolan.org/listinfo/vlc-devel
>>
>
> --
> Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma brièveté.
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20190321/92256035/attachment-0001.html>
More information about the vlc-devel
mailing list