[vlc-devel] Smooth Streaming manifest, track IDs

[Frederic YHUEL]

On Fri, Feb 17, 2012 at 10:58 AM, Martin Storsjö <martin at martin.st> wrote:
> On Fri, 17 Feb 2012, Frederic YHUEL wrote:
>
>> Hi Martin, thanks for your help!
>>
>> On Fri, Feb 17, 2012 at 8:30 AM, Martin Storsjö <martin at martin.st> wrote:
>>>
>>> Hi Frederic,
>>>
>>>
>>> On Wed, 15 Feb 2012, Frederic YHUEL wrote:
>>>
>>>> Why does the (client) manifest *not* contain the track IDs?
>>>>
>>>> I guess that what Microsoft had in mind, when they wrote the specs, is
>>>> that the client can associate an arbitrary ID to each track. Since the
>>>> client does know from which track belongs each chunk, all should go
>>>> well.
>>>
>>>
>>>
>>> I guess the reason is that it normally isn't required - each fragment
>>> only
>>> contains data from one single track anyway.
>>
>>
>> Ok
>>
>>> (And within one smooth streaming
>>> presentation, the tracks might either reside in separate ismv files and
>>> thus
>>> have overlapping track ids, or they might all be within the same and have
>>> unique ids.)
>>>
>>
>> Overlapping track IDs, really? Oh that sucks, at least that means that
>> my "pre-fetching" solution won't always work... :-/ Though I have the
>> feeling that in the vast majority of cases, we have one and only one
>> ismv file per quality level, and thus unique IDs.
>
>
> Hmm, I'm not sure if we mean the same or different things here. What I meant
> was that since you have one ismv file per quality level, all the quality
> levels might have trackID=1 - there's nothing stopping that from happening.
> And if the audio stream is in a different file, that one might just as well
> have trackID=1 too.
>

Yes I think we mean the same thing. I think I understood well.

>
>>>
>>>> But the way VLC is done makes things tricky. Retrieving the "real"
>>>> track IDs is compulsory, if we use avformat demux, and is very helpful
>>>> otherwise (I'm working on VLC's MP4 demux).
>>>
>>>
>>>
>>> Why do you need knowing the real track id if using the avformat demuxer?
>>
>>
>> I don't really know how the avformat demuxer works, but it doesn't
>> know, a priori, what data it gets from the stream_filter module. In
>> particular, for a given fragment, it cannot guess which track it
>> belongs to, without looking at the ID in the tfhd box.
>>
>> When I was using it, I used to generate first a moov box from the
>> manifest, in which I used to put the "real" track IDs in each tkhd
>> box. It used to work well, if and even if track IDs of the moov box
>> were matching those in fragment's tfhd boxes.
>
>
> Ah, yes, if you use the demuxer that way (no, I don't know of any better way
> of tricking the lavf mov demuxer to parse plain fragments), then you will
> need to know the track ID beforehand I guess, and some kind of prefetch
> probably is the only way.
>
> Another alternative would be to change the demuxer to be able to handle
> plain fragments without any moov header, but that then requires a way of
> initializiting the demuxer.
>

I'm doing that with VLC MP4 demux, but still, it is useful to be able
to know which track a fragment belongs to, without using out of band
communication between the demux and the stream_filter.

> Yet another alternative would be to make up your own track IDs, and rewrite
> the fragments that you pass to the demuxer to match the track IDs you've
> chosen. Not pretty of course, but might not be insanely complex either.
>

Yes, indeed! This is the solution! This is not insanely complex, and
it's guaranteed to work. I should have think about it... Thanks!

>
>>> If
>>> you use one separate demuxer for each fragment, there shouldn't be any
>>> ambiguity, but does that have other drawbacks?
>>>
>>
>> s/fragment/track ?
>
>
> Yes, sorry.
>
>
>> Maybe I could have done that, but I don't know how.
>>
>>>
>>>> Do you think my guess is correct? Or maybe there is a simple way to
>>>> retrieve the track IDs after all? The only way I have in mind, is to
>>>> pre-fetch the first (lowest quality) chunk of each track, and to parse
>>>> the tfhd atom to finally get the track ID.
>>>
>>>
>>>
>>> Also, a small note about prefetching (which might be informative but
>>> probably not helpful): I guess you know that you can request
>>> QualityLevels(...)/FragmentInfo(...) instead of Fragments. This gives you
>>> only the moof atom, not the mdat part. This is used by the silverlight
>>> player to fetch info about upcoming fragments (that are advertised within
>>> the tfrf uuid tags) in live streams, without downloading the full
>>> fragments.
>>>
>>
>> Cool! No I didn't know that... A few months ago, I didn't find
>> [MS-SSTR], all what I found was the old [MS-SMTH] paper... the SS
>> documentation is quite scatterd, or maybe I'm just stupid.
>
>
> It's really scattered, and the documentation doesn't really explain stuff,
> it just states things without any explanation of how it all fits together.
>

Ok I'm relieved, it's not just me :-)

>
>>> This is possible, since even if the client manifest explicitly contains
>>> declarations like
>>>
>>> Url="QualityLevels({bitrate})/Fragments(video={start time})",
>>>
>>> the spec actually mandates this to be almost literally this string,
>>> there's
>>> not much variability. In particular, the "Fragments" string must be
>>> exactly
>>> that, since apparently the silverlight players can swap it out against
>>> FragmentInfo. The spec also mentions a few other similar things that can
>>> be
>>> requested.
>>>
>>> I'm not sure if all smooth streaming content providers actually answer to
>>> requests for FragmentInfo though, it might have been omitted in some
>>> implementations of static content I think, since the official silverlight
>>> player only requests it for live streams.
>>>
>>
>> I got a MPD with that:
>> Url="video_bitrate_{bitrate}_PTS_{start time}.ismv"
>>
>> But at the same time, this Smooth Streaming server is weird, since
>> there is one unique track ID per track *and per quality level*
>
>
> Hmm, that url doesn't seem valid according to how I read the specs at least.
> From [MS-SSTR], 2.2.2.4:
>
> UrlPattern = QualityLevelsPattern "/" FragmentsPattern QualityLevelsPattern
> = QualityLevelsNoun "(" QualityLevelsPredicatePattern ")"
> QualityLevelsNoun = "QualityLevels"
> QualityLevelsPredicate = BitrateSubstitution [","
> CustomAttributesSubstitution ]
> Bitrate = "{bitrate}" / "{Bitrate}"
> CustomAttributesSubstitution = "{CustomAttributes}"
> FragmentsPattern = FragmentsNoun "(" FragmentsPatternPredicate ")";
> FragmentsNoun = "Fragments"
> FragmentsPatternPredicate = TrackName "=" StartTimeSubstitution;
> TrackName = URISAFE_IDENTIFIER_NONNUMERIC
> StartTimeSubstitution = "{start time}" / "{start_time}"
>
> As far as I interpret this, there's not much else you can vary freely than
> TrackName and CustomAttributesSubstitution.
>

Indeed... it seems to me that each Smooth Streaming server has its
weirdness, depending of the version of MS Expression Encoder used, or
their own encoder...

Anyway, you gave me a good solution to my problem, and I don't need
anymore pre-fetching.

Best Regards,

-- 
Frédéric