[vlc-devel] [PATCH 2/3] vout: pause and resume the decoder pool

[Rémi Denis-Courmont]

On Friday 27 November 2015 17:28:56 Thomas Guillem wrote:
> > I am not sure this problem is even specific to pause, and not just pure
> > timing chance.
> > 
> > In any case, that looks like a problem with flushing, not with pausing.
> > There was previously a kludge (I think by "the other" Felix) to prevent
> > picture allocation during flush.
> 
> Yes, I know this problem, I reworked the hack in order to don't depend
> on Decoder_IsFlushing (that is dead now). But now, the problem occurs
> only with avcodec (on any platforms).

I think checking the flushing flag is unworkable at this point:  the clearing 
the decoder flushing flag and flushing the decoder would have to be done 
atomically.

On the one hand, currently, the flushing flag is cleared after the decoder is 
flushed. A threaded decoder could attempt to allocate a picture buffer in the 
interval, e.g. processing extra data, or just in anticipation. So allocating 
the picture buffer cannot fail while flushing: That would lead to spurious 
decoding errors, and potentially loss of key frame.

On the other hand, if the flushing flag were cleared before flushing the 
decoder, a race condition would exist whereby the decoder attempts to allocate 
a picture buffer before the flush callback - leading back to the original 
problem of waiting on the pool.

So I can only think of two remaning avenues:
- flush the video output before *and* after the decoder,
- make the decoder plug-in responsible for aborting picture allocation on 
flush (seems impractical though).

> > I don't think anybody ever figured out what the real problem was
> > though. As far as I can tell, there are no reasons why we should run out
> > of picture buffers specifically during flush. I do not even know how to
> > reproduce the lock-up.
> 
> To reproduce it: any h264 video decoded with avcodec with mt. Play,
> Pause, Stop -> deadlock (Play, Pause, Seek with my avcodec flush patch).

That works just fine here.

-- 
Rémi Denis-Courmont
http://www.remlab.net/