[vlc-devel] vlc-devel Digest, Vol 94, Issue 42

[Goldy Liang]

Hi Rémi,

Thanks for the response and it is very inspiring. Please see inline
comments below and my new ideas.

Goldy




> Date: Wed, 25 Mar 2015 19:08:55 +0200
> From: R?mi Denis-Courmont <remi at remlab.net>
> To: vlc-devel at videolan.org
> Subject: Re: [vlc-devel] [PATCH 1/2] Navigation enhancement for
>         language        learning
> Message-ID: <14274086.51sHASZh0G at basile.remlab.net>
> Content-Type: text/plain; charset="utf-8"
>
> > On Thu, Mar 19, 2015 at 6:13 AM, Goldy Liang <goldyliang at gmail.com>
> wrote:
> > > I just had a glance how EsOutSend works, but I don't think it is good
> to
> > > implement there. Looks like EsOutSend would be called by all kinds of
> > > modules of demuxer or access. In EsOutSend, if I want to exact the time
> > > for subtitle, I have to check whether it is sent by the subtitle
> modules.
>
> I am not intimate with the ES output, but the type of the track, and its
> active status should be known there.
>
> > > And looks like each subtitle demux will send the data anyway no matter
> if
> > > it is selected or default.
>
> >From a quick look, the track is active if it has a non-NULL decoder_t
> pointer
> - but again, I am not an expert with es_out.c.
>
> It does not matter anyway. You can collect the timings for inactive tracks
> too. That way, seeking will work even if the user just changed the subtitle
> track. It is only when actually seeking that the input thread need to know
> which track is active, to query the timings from the correct track.
>

[Goldy]  If we want to seek a universal solution, yes EsOutSend may work.
However looks like DecoderProcessSpu could be better. If my understanding
is right, it should be called by any kinds of subtitle, then a specified
decoder is called for different types by calling "p_dec->pf_decode_sub".
(Correct me if I am wrong).

So the idea could be that, the decoder keeps and caches all the decoded
subtitle timing information on the fly, so that the input thread is able to
query the decoder module to get the timing of a prev/current sentence. Or
even the timing of the next sentence (if it is ever decoded). The input
thread could query the decoder to dump all cached subtitle data as well
(for future use).

This solution should work for most (if not all) subtitles, even if it has
the limitation like:
   - skip to next sentence may not work.
   - skip to previous sentence may not work either, if the user has just
done a forward jump (information of prev sentence not known)


> > > Even if we can identify the subtitle data and save it, it all depends
> how
> > > much data has been sent so as to navigate based on that.
>
> Yes. There would be a problem if a sub.itle demuxer sent a single subtitle
> as
> multiple blocks. But then again, the ES output works with all file formats
> and
> all codecs.
>
> It will be very difficult to explain users that the feature only works
> with some
> subtitle formats but not all, if we go with the subtitle.c solution.
>

[Goldy] It is definitely good if the function can work for all subtitle
formats.

However besides working on the universal solution, I would still want to
keep the subtitle.c
solution because the stand-alone text-based subtitle files have the least
limitation for
subtitle based navigation, and I believe this kind of subtitle resources
are very common
in the internet. So, if there is subtitle loaded by subtitle.c, totally
perfect, if not, the universal
solution still works for most functions.


> > > For example, if the data of next sentence not sent yet, we can not jump
> > > forward one sentence; or if the user just changes the slider bar and
> jump
> > > to a specific position, we may be missing the time data for the
> previous
> > > sentence, etc. And another problem is where to save the time data and
> how
> > > the input thread can access it. In a word, it looks complicate and not
> > > good to me.
>
> Jumping forward is not possible as such with the ES output. Still fast
> forward
> would be feasible if a little difficult. Also, fast forwarding is usually
> more
> robust than seeking.
>

[Goldy] Doing a fast forward would be a good idea. It looks a bit hard but
obviously workable.
Also, if a prev-sentence information is not there, a fast backward should
also work.


>
> I would however think that seeking to previous and current subtitles is
> much
> more essential to the intended use case than seeking to the next subtitle.
>

[Goldy] When the user wants to review a video clip for language study,
seeking forward is a
useful function.

I think I can use the interface I mentioned above to dump the cached
subtitle information to the interface
module, then the interface can save it in a standalone subtitle file which
can be used later on for navigation
purpose.


>
> > > On the other hand, by controlling the demux, it is more simple and
> > > provides more flexibility for future development.
>
> Controlling the demux is likely simpler if you stick to subtitle.c. But the
> amount of changes in src/input/ as a whole is comparable if not larger.
>
> Still, the big problem with the demux approach is tht it will affect a lot
> of
> different demux plugins. On top of my head, besides subtitle, avformat,
> AVI,
> dvdnav, dvdread, MKV, MP4, MPEG-PS, MPEG-TS can contain subtitles, and
> that is
> not even exhaustive.
>
> > > I have more features in my mind about language learning, for example,
> we
> > > can show all subtitles in a separate window so that the user can glance
> > > and jump to any sentence he wants.  This requires the interface module
> to
> > > be able to access the whole subtitle data instead of just the time of
> > > prev/next sentence. I would like to make the subtitle demux to clone
> and
> > > dump the whole subtitle data through the input thread control to the
> > > interface. That would be consistent with the current mechanism that the
> > > subtitle demux reports the timestamps of prev/cur/next sentences to
> input
> > > thread then to interface. If we do that in es_out, I think it would
> not be
> > > possible to allow the interface to access the subtitle data freely.
>
> The least intrusive way to dump the whole subtitle track for display and/or
> random access ahead of time in a format-independent fashion is to run the
> demuxer in a tight loop with a purpose-specific es_out_t instance that just
> records all the subtitles for later use and discards everything else.
>
> An example of custom es_out_t is the Bluray plugin.
>

[Goldy] Dumping the subtitle data like your said would be a next-step for
me.
However dumping the subtitle data loaded in subtitle.c would be more easy
for
now.


>
> --
> R?mi Denis-Courmont
> http://www.remlab.net/
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20150329/c3cccbd6/attachment.html>