[vlc-devel] [PATCH 0/6] Trying to fix the big one: Frame threading regressions

[Thomas Guillem]

Hello,

And again, a new branch here: https://code.videolan.org/tguillem/vlc/tree/decoder-frame-delay/3 <https://code.videolan.org/tguillem/vlc/tree/decoder-frame-delay/2>

I attached the diff between this new branch and my second branch.

Changes since the second branch:
 - Renamed decoder_UpdateVideoFrameLatency() to decoder_UpdateVideoLatency() that takes now a latency in argument (no more frame_count)
 - Handle fps that is only known from an avcodec thread (from the ffmpeg_GetFormat() callback): In that case, the module can't handle the decoder_UpdateVideoLatency() failure and will continue anyway (after displaying a warning). This should be a corner case. To mitigate this case, packetizers should parse the frame_rate for more codecs (already the case for hxxx, not done for AV1).

On Fri, Sep 27, 2019, at 11:45, Thomas Guillem wrote:
> Here is new a branch https://code.videolan.org/tguillem/vlc/commits/decoder-frame-delay/2
> 
> I attached the diff between this new branch and my first branch (grrr, impossible to print a diff from 2 non-master branches from gitlab)
> 
> Changes since the first branch:
> 
> - The decoder latency is not added to the overall clock dejitter delay. Only the MAX between the input_dejitter (file/network-caching) and the highest decoder latency will be used. The update of the decoder latency will always succeed if it is lower than input_dejitter. For example, using the default file-caching of 300ms, video decoders will always be able to setup 9threads at 30fps (1000*9/30 = 300). If a video decoder need more threads or a lower fps, the decoder has to be started just after/before (80ms gap) the audio decoder. 
> 
> - The avcodec module will try to setup less thread instead of fallbacking directly to 1 in case of decoder_UpdateVideoFrameLatency() failure.
> 
> Note: Videos tracks will very likely be started just after/before the audio (80ms gap). In that case, the update of the decoder latency will always succeed regardless of the fps/theead_count.
> 
> On Thu, Sep 26, 2019, at 16:38, Thomas Guillem wrote:
> > cf. https://mailman.videolan.org/pipermail/vlc-devel/2019-September/127432.html
> > 
> > I chose to solve this issue with:
> > "- Have a way to report decoder delay.. but that would mean no playback
> > until data decodes (what if only bogus data ? mutiple decs ?..)."
> > 
> > Yes, no playback until data decodes, that is why we have a jitter delay in the
> > output clock (80ms minimum).
> > 
> > The solution I propose (explained commits by commits) will work on most case
> > but won't work if the video is started late. In that case, threaded decoding
> > will be disabled by avcodec. I guess it's OK since it's a corner case.
> > 
> > Thomas Guillem (6):
> > clock: remove unused variable
> > clock: store the full list of clocks on the main one
> > clock: add vlc_clock_SetDecoderLatency
> > decoder: add decoder_UpdateVideoFrameLatency
> > avcodec: re-indent for the next commit
> > avcodec: use decoder_UpdateVideoFrameLatency()
> > 
> > include/vlc_codec.h | 22 +++++++++
> > modules/codec/avcodec/video.c | 69 ++++++++++++++++++---------
> > src/clock/clock.c | 88 ++++++++++++++++++++++++++++++++++-
> > src/clock/clock.h | 13 ++++++
> > src/input/decoder.c | 30 ++++++++++++
> > 5 files changed, 197 insertions(+), 25 deletions(-)
> > 
> > -- 
> > 2.20.1
> > 
> > _______________________________________________
> > vlc-devel mailing list
> > To unsubscribe or modify your subscription options:
> > https://mailman.videolan.org/listinfo/vlc-devel
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
> 
> *Attachments:*
>  * decoder-frame-delay.diff
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20190930/1f4b57fc/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: decoder-frame-delay.diff
Type: text/x-patch
Size: 9186 bytes
Desc: not available
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20190930/1f4b57fc/attachment.bin>