[vlc-devel] [RFC PATCH] mediacodec: fix a deadlock when seeking on pause

[Martin Storsjö]

On Tue, 28 Oct 2014, Thomas Guillem wrote:

> MediaCodec with direct rendering can be stuck waiting for an input buffer when
> video is paused. Indeed, input buffers availability depends on output buffers.
> When the video is paused, output buffers are not released or displayed anymore,
> therefore, no more input buffers are available.
>
> A previous hack was made in order to fix a deadlock when mediacodec was stuck
> and when exit was requested. Mediacodec returned an invalid picture after a
> small delay, then DecoderIsExitRequested in DecoderDecodeVideo was checked and
> this decoder could be exited without blocking.
>
> This previous hack didn't fix a deadlock when a paused video was flushed, due
> to a seek. Indeed, when stuck, mediacodec is feed with the same block.
> Therefore it won't receive a new block containing the FLUSH flags and it will
> be stuck returning an invalid picture.
>
> One (ugly) way to fix this deadlock is, when you return an invalid_picture, to
> put the block in an internal FIFO and tell the core decoder that the block is
> processed. This way, new blocks will be given on next Decode calls and the
> Decoder will be able to handle a flush block. If the decoder is stuck when
> paused and resumed again, the Decode function will first process the block that
> come from the internal FIFO. There will be no discontinuity at all.

I'm a little hesitant. Doesn't this mean that while paused, the decoder 
would (slowly) buffer up all later video packets in the file? So if you 
keep the player paused for a little longer while, you'd end up with almost 
the whole file in memory? And once you start playing, there's no way to 
reduce the length of the fifo (you only remove one packet from the FIFO 
when you get a new one) other than seeking, or am I missing something?

// Martin