[vlc-devel] [PATCH] [RFC] youtube.lua: read more available formats from the webpage

[Steve Lhomme]

On 2020-09-29 12:55, Pierre Ynard via vlc-devel wrote:
>> It seems that "formats" is only providing 2 basic versions of the
>> video in H264 in my tests: 720p and 360p
>> (https://www.youtube.com/watch?v=ISJMsVhY5Yk or https://www.youtube.com/watch?v=ISJMsVhY5Yk)
> 
> Same URL twice

Oops, yes. Anyway, most URL should do the same (live streaming is 
probably different as we should access the real manifest).

> The attached patch indeed causes a video-only very HD video to be
> played, without sound.
> 
> This is a known limitation: https://trac.videolan.org/vlc/ticket/10237
> 
> The adaptiveFormats field does provide a much wider range of available
> formats, that are video-only or audio-only and not directly usable. I
> guess that even with the current API, we could pick one of each and play
> them both using an input-slave option, but that means potential for
> synchronization and playback issues that we avoid with a single muxed
> stream, and twice the network responsiveness issues - all the more that
> these HD streams will be more demanding on the network and more likely
> to cause lag, buffering and stalls. These are reasons why I haven't
> pursued it.

The input slave (!) is a possibility although that means hardcoding the 
decision when the playback starts. I think we would better off with 
providing all URLs to our adaptive streaming stack and it would take the 
decision on what to use. The same could be done for other sources than 
Youtube.

>> "adaptiveFormats" provide a lot more formats, including 8K AV1 ones.
>> That may cause some problems as the hardware may not be able to decode
>> such streams. We're probably missing some more refinements to select
>> the best resolution/bitrate that the hardware can handle.
> 
> When --preferred-resolution was introduced, it was expected to be
> a non-issue most of the time, as 1080p streams weren't too widely
> available and it was expected that the average setup would be able to
> play the highest resolution available in most cases. The default setting
> was and still is to pick the highest resolution available.
> 
> Now we're talking about putting VLC in a situation where the average
> setup might be challenged by default. We can change the default to limit
> at 1080p for example, or add hardware detection magic, but ultimately
> options that are tucked away in a corner of the advanced preferences
> might not be an acceptable answer. Which leads to...
> 
>> I would assume that's something adaptive streaming is best suited to
>> take care of. So maybe the Youtube script should return an adaptive
>> manifest and let the adaptive code take care of it ? Is that even
>> possible ?
> 
> Maybe we should have something like that, if it works fine; if it was
> less reliable than muxed streams then there would be a trade-off to
> doing that. But I think we'd also need a UI component to control this
> adaptive resolution (or even enable it).

If adaptive streaming is inefficient then we already have a problem with 
a lot of sources (including live youtube streams ?). I think that's the 
logical way.

> I don't know based on what the adaptive demux currently adapts the
> resolution. I know it supports a bunch of quality settings; however it
> ignores --preferred-resolution (#21828), which doesn't help these things
> coexist and is a current UX bug with YouTube live streams - and also
> with Dailymotion which switched exclusively to adaptive manifests. An
> easy first step would be to fix what we already have.

Yes, supporting "preferred-resolution" in there should be done (poke 
Francois).

> Have you tried picking a pair of audio-only and video-only YouTube
> stream URLs and writing by hand a sample adaptive manifest file, and
> having VLC play it? How does that work?

I tried turning the Youtube stream filter "playlist/directory" into a 
stream filter that reads the YT data and sends HLS data in return. So 
far it's unsuccessful because the "data" stream filter is opened many 
times for reading from the same HTTP stream (the YT page) many times. 
And only the first one gets proper data. Once I figure out why I'm 
confident this can work in the end.