[vlc-devel] [PATCH] video_output: wait half the extra time we have without getting the display_lock

[Steve Lhomme]

On 2/19/2021 4:06 PM, Rémi Denis-Courmont wrote:
> Le perjantaina 19. helmikuuta 2021, 15.12.49 EET Steve Lhomme a écrit :
>> In an ideal world we would never wait between prepare and display, they
>> would have predictable time and we could estimate exactly when we must
>> do the prepare to be on-time for the display.
> 
> That's not ideal either. prepare() should rather be called at the earliest
> option, which is to say whence the picture has been filtered and converted.
> Estimations should only be used for display() if at all.

I disagree. First the picture is not "filtered" at this stage, only 
deinterlaced and converted. The user filters are applied just before 
rendering. That allows to be more responsive to user changes (or if the 
filter has some internal timing). And also take the latest 
mouse/viewpoint values in account.

And because of that the user-filtering and rendering needs to be as 
close as possible to the moment it will be displayed. That allows 
minimizing the lag perceived between the user input and the display.

This patch helps with the UI responsiveness, but not with the input lag. 
The wait is done in the code that already has the mouse settled. The 
mouse values won't change anymore (the viewpoint can).

The wait would be better done in the part that waits for mouse events 
until the given deadline. Except it doesn't work well on Windows because 
of deadline not working well.

Maybe if the mouse was handle like the viewpoint we could have more 
mouse responsiveness (on the rendered data, not UI resizing and such).

> This is really a design flaw in the the interface exposed by the "vout display"
> capability. For a start, it should be possible not to display a picture that
> has been prepared. Eventually it should also be possible to prepare multiple
> pictures ahead. And ultimately, it should maybe be possible to display the
> same picture multiple times, though that last point is much less useful now
> that compositing is pervasive.
> 
> We already talked about that problem at the last vout workshop two years ago.
> 
>> That would also leave all that time to the vout control loop to process
>> more mouse events that might occur before we start the displaying.
> 
> Unfortunately, it is not that easy. Once the display lock is released, the
> display might reset (especially the dumb ones like X11 and GDI), and
> invalidate the prepared picture. With the current interface, we cannot unlock
> the display between prepare() and display().
> 
> Without question, we should allow it. It's not just mouse events, but all
> interactions with the windowing system. In particular, we should forcefully
> redraw the last picture immediately  whenever the display "damaged" (in the
> windowing system sense).
> 
> But again, the current vout display capability has no such provisions.
>