[vlc-devel] [PATCH] video_output: wait half the extra time we have without getting the display_lock

[Steve Lhomme]

On 2/20/2021 10:22 AM, Rémi Denis-Courmont wrote:
> Le lauantaina 20. helmikuuta 2021, 11.03.18 EET Steve Lhomme a écrit :
>> On 2/19/2021 4:06 PM, Rémi Denis-Courmont wrote:
>>> Le perjantaina 19. helmikuuta 2021, 15.12.49 EET Steve Lhomme a écrit :
>>>> In an ideal world we would never wait between prepare and display, they
>>>> would have predictable time and we could estimate exactly when we must
>>>> do the prepare to be on-time for the display.
>>>
>>> That's not ideal either. prepare() should rather be called at the earliest
>>> option, which is to say whence the picture has been filtered and
>>> converted.
>>> Estimations should only be used for display() if at all.
>>
>> I disagree. First the picture is not "filtered" at this stage, only
>> deinterlaced and converted.
> 
> That's plain nonsense. At the stage "whence the picture has been filtered and
> converted", the picture has, by definition, been filtered.
> 
>> The user filters are applied just before rendering.
> 
> That's a bug. It prevents evening out the resources usage (both internally and
> externally with other concurrent processes), and generally being more robust
> against scheduling glitches. It's also incompatible with modern video filters
> and outputs that support queueing.
> 
> There is no basis to assume that only deinterlacing requires ahead-of-time
> processing for optimal results. Motion blur is an obvious counter example
> here. Besides, in practice deinterlacing is the only filter (if any) in most
> cases.

There's a very simple scenario to support this behaviour. Pause the 
video and change the user filters from the UI. The render changes. It's 
done on the same "pre-rendered" (deinterlace+convert) picture.

> And indeed, audio filters don't have that bug. They filter as far ahead as
> possible. How long that is depends on the low latency setting. Not every case
> is treated as low latency just because.
> 
>> That allows to be more responsive to user changes (or if the
>> filter has some internal timing).
> 
> That is optimising for an exceptional case (user interaction) within an
> exceptional case (filters active other than deinterlace). It makes zero sense,
> especially if it hurts performance and robustness.
> 
> Besides, the plan from the clock/buffer rework was pretty much for version 5.0+
> to stop relying on the clock for pacing/buffering and instead push buffer as far
> as possible.
>