[vlc-devel] Handling deadlock between display:Open() and vout_window_ReportSize()

[Alexandre Janniaux]

Hi,

On Sun, Dec 15, 2019 at 01:37:24PM +0200, Rémi Denis-Courmont wrote:
> Le sunnuntaina 15. joulukuuta 2019, 13.20.43 EET Alexandre Janniaux a écrit :
> > Hi,
> >
> > After some thoughs, surveys of locks and schematics I might
> > have a solution for allowing the vout_display::Open to block
> > the main thread without additional lock.
> >
> > It seems that I can just move the display creation outside of
> > the display_lock, and then lock it to share it with sys while
> > applying the pending requested state after the Open() has
> > ended, so that the behaviour stays the same.
>
> That breaks the lock chaining which ensures that the configuration stays in in
> synchronization in the window, display and core.

Isn't it a false synchronization, if you have a race between
display open and resize? With this change you would have
either:

+ the display is created before the resize, the resize will
  update display size through control.
+ the display is created after the resize, the configuration
  is correct from the beginning.
+ the display is created at the same time as the resize, it
  opens with the previous configuration and will get a resize
  control in the display lock right after it opened. Same for
  other state changes.

> Besides, it solves exactly nothing. You still need to lock the display for
> pretty much every other calls, which leads to the same deadlocks.

As far as I can see in other plugins, other calls can be
implemented as asynchronous. As I explained in the first post,
my main concern is only about Open() which must return a
correct return value.

> And further besides, you _cannot_ block on the application main loop from
> output plugins of any kind, as it causes recursion or dead lock when the
> application invokes LibVLC functions. This has nothing to do with the video
> output, really.

Sorry this was confusing, I meant waiting for the main loop
to process the needed event for the display Open to complete.

I agree with you.

The issue I described is actually litterally that we're
blocking the main thread because of the vout code, and I'm
trying not to.


Regards,
--
Alexandre Janniaux
Videolabs