[vlc-devel] Chromecast design

[Steve Lhomme]

I am currently working on the Chromecast plugin(s) for VLC. There's
currently a plugin that is a sout wrapper but it assumes the
Chromecast is "playback slave" and VLC is the master. In reality this
is not how the Chromecast works. The Chromecast is the player with its
own buffering, timing and multiple controllers possible
(play/pause/exit).

Because the current solution is a sout, it doesn't know how play/pause
events. It also doesn't know about file changes and thus doesn't allow
us different strategies to provide the data to the Chromecast
(transcoded, remuxed, file served as-is over HTTP, remote HTTP URL
given directly to the Chromecast).

Because the Chromecast is the player, in some of these scenarii, VLC
becomes a slave (controlled) by the Chromecast player. For example
when serving a muxed file via HTTP, the Chromecast will buffer a lot
of data in advance (20s). The VLC code is currently not designed to
handle that. It assumes the playback time currently in use is the one
provided by the demuxer (if I'm not mistaken), resulting in an
overhead of 20s and also ending the playback 20s too early.

So the idea is to move the communication with the Chromecast out of
the sout so we can have control/access to the play/pause/open/end
events from the player. A control interface is the most logical with
the current code. This way we every URI we try to play we can
configure the player to output the right data for the Chromecast and
also tell the Chromecast to start/stop reading that stream. This part
is working well already.

The trickier part is that 20s delay (may differ depending on the
bitrate of the output stream) and also how we can eventually handle
remote URLs that the Chromecast can playback on its own. These are 2
separate problems but I think they are related. Also it should be
better to go right away with a solution that can handle remote URLs.

I think a "demuxer wrapper" could be a solution since that's where the
time reported is handled. Whenever the Chromecast starts/pauses/exist
playback, it broadcasts the event to all clients connected (VLC and
the Chromecast Android app for example). We could use that as a time
base and use a monotone clock whenever we're playing and asked the
current playback time. We don't need to query the Chromecast every
time.

There's still an issue at the end of the file though. The way
MainLoop() in input.c currently works is that when the input source
reaches EOF, it waits until es_out is empty and then loads the next
file. Being mostly a slave here, we need to wait until the file is
actually finished sending (almost at the same time) and played
(happens 20s after our EOF). I think the condition on the empty es_out
doesn't work for us as it's empty before transcoding or muxing and
thus far before playback is finished. So I think we need an extra
condition in there to wait until we start playing the next file. I
don't know what form it could take yet. It could be another layer of
es_out wrapping like currently done with time shifting but for
Chromecast, or something more generic. This part is currently hard
coded in the core.

Or is there a mechanism I missed that already ensures that all data
are played ? I suppose that's needed for gapless playback. In that
case you want to continue playing the previous file while already
starting decoding the new file. This is trickier in the case of this
sout because not only you need to wait until the buffers are all sent,
but also that the Chromecast has used them. This is another reason why
a pure sout solution cannot work, it has no notion of end of file.

As for the remote URLs, if we go with hacking using a monotone clock
to mimick the Chromecast time and wait for the Chromecast to signal us
it's finished playing, we could force some dummy demuxer for that kind
of playback so it thinks it's ready data when in fact it isn't. That
would likely involve a dummy "access_demux" that would never actually
push any data downstream.

To summarize we have the following problems:
1) report the playback time in the UI correctly.
2) handle the EOF with the long playback delay due to the Chromecast buffering.
3) handle remote URLs by pretending we're reading and sending data.

That involves a lot of hacking/changes in the core to be able to
handle the Chromecast properly. So I need some feedback on which way
would be okay or not.