[vlc-devel] [PATCH] decoder: fix pause+close deadlock for decoders waiting internally for buffers

[Steve Lhomme]

On 2020-07-29 22:28, Rémi Denis-Courmont wrote:
> Le keskiviikkona 29. heinäkuuta 2020, 11.04.13 EEST Steve Lhomme a écrit :
>> When paused the decoder decodes until it exhausts its available buffers then
>> waits until new buffers are available. When seeking we force flushing the
>> vout which releases the buffers the decoder is waiting for.
>>
>> When closing while pause we didn't flush the vout so the decoder was still
>> waiting for buffers even though it's not going to use them (or display
>> them). So we do the same flushing to automatically release the used/pending
>> buffers. ---
>>   src/input/decoder.c | 16 ++++++++++++++++
>>   1 file changed, 16 insertions(+)
>>
>> diff --git a/src/input/decoder.c b/src/input/decoder.c
>> index 141120d6395..464ceff046e 100644
>> --- a/src/input/decoder.c
>> +++ b/src/input/decoder.c
>> @@ -2153,6 +2153,22 @@ void vlc_input_decoder_Delete( vlc_input_decoder_t
>> *p_owner ) {
>>           if (p_owner->out_pool)
>>               picture_pool_Cancel( p_owner->out_pool, true );
>> +
>> +        if( p_owner->paused )
> 
> I don't exactly follow why this check is needed here, nor in the existing flush
> code. It's debatable whether we should flush on delete, but assuming so, then I
> don't see what pause has to do with it, TBH.

I just copied the part done in vlc_input_decoder_Flush(). It should work 
in non pause as well, during the delete. But IMO it's better to signal 
it's only needed for that reason. This is not "push model" IMO. We 
should not depend on the vout unblocking us, but that's what we have for 
now. I'd rather the normal flow be used when we don't need this hack. 
But I don't have a strong feeling about this. The comment could be enough.