[vlc-devel] [PATCH 1/2] Navigation enhancement for language learning

[Rémi Denis-Courmont]

> On Thu, Mar 19, 2015 at 6:13 AM, Goldy Liang <goldyliang at gmail.com> wrote:
> > I just had a glance how EsOutSend works, but I don't think it is good to
> > implement there. Looks like EsOutSend would be called by all kinds of
> > modules of demuxer or access. In EsOutSend, if I want to exact the time
> > for subtitle, I have to check whether it is sent by the subtitle modules.

I am not intimate with the ES output, but the type of the track, and its 
active status should be known there.

> > And looks like each subtitle demux will send the data anyway no matter if
> > it is selected or default.

>From a quick look, the track is active if it has a non-NULL decoder_t pointer 
- but again, I am not an expert with es_out.c.

It does not matter anyway. You can collect the timings for inactive tracks 
too. That way, seeking will work even if the user just changed the subtitle 
track. It is only when actually seeking that the input thread need to know 
which track is active, to query the timings from the correct track.

> > Even if we can identify the subtitle data and save it, it all depends how
> > much data has been sent so as to navigate based on that.

Yes. There would be a problem if a subtitle demuxer sent a single subtitle as 
multiple blocks. But then again, the ES output works with all file formats and 
all codecs.

It will be very difficult to explain users that the feature only works with some 
subtitle formats but not all, if we go with the subtitle.c solution.

> > For example, if the data of next sentence not sent yet, we can not jump
> > forward one sentence; or if the user just changes the slider bar and jump
> > to a specific position, we may be missing the time data for the previous
> > sentence, etc. And another problem is where to save the time data and how
> > the input thread can access it. In a word, it looks complicate and not
> > good to me.

Jumping forward is not possible as such with the ES output. Still fast forward 
would be feasible if a little difficult. Also, fast forwarding is usually more 
robust than seeking.

I would however think that seeking to previous and current subtitles is much 
more essential to the intended use case than seeking to the next subtitle.

> > On the other hand, by controlling the demux, it is more simple and
> > provides more flexibility for future development.

Controlling the demux is likely simpler if you stick to subtitle.c. But the 
amount of changes in src/input/ as a whole is comparable if not larger.

Still, the big problem with the demux approach is tht it will affect a lot of 
different demux plugins. On top of my head, besides subtitle, avformat, AVI, 
dvdnav, dvdread, MKV, MP4, MPEG-PS, MPEG-TS can contain subtitles, and that is 
not even exhaustive.

> > I have more features in my mind about language learning, for example, we
> > can show all subtitles in a separate window so that the user can glance
> > and jump to any sentence he wants.  This requires the interface module to
> > be able to access the whole subtitle data instead of just the time of
> > prev/next sentence. I would like to make the subtitle demux to clone and
> > dump the whole subtitle data through the input thread control to the
> > interface. That would be consistent with the current mechanism that the
> > subtitle demux reports the timestamps of prev/cur/next sentences to input
> > thread then to interface. If we do that in es_out, I think it would not be
> > possible to allow the interface to access the subtitle data freely.

The least intrusive way to dump the whole subtitle track for display and/or 
random access ahead of time in a format-independent fashion is to run the 
demuxer in a tight loop with a purpose-specific es_out_t instance that just 
records all the subtitles for later use and discards everything else.

An example of custom es_out_t is the Bluray plugin.

-- 
Rémi Denis-Courmont
http://www.remlab.net/