[vlc-devel] vlc-devel Digest, Vol 94, Issue 42

Mon Mar 30 06:50:14 CEST 2015

For following comment, I think you meant that subtitle demux is not
thread-safe. Because I tried to call demux_Control by the input control,
which is called by the interface thread?

I know the concern here. Actually I already tried calling input_ControlPush
with a command similar to INPUT_ADD_SUBTITLE/INPUT_CONTROL_ADD_SUBTITLE.
But it failed because I want the control to return the time value. By
calling input_ControlPush, the handling is asynchronous, so that the
function call can not get the value right away. I don't know how to solve
that. Not sure if there is any other example to return a value
asynchronously?

> +
> +           if (!p_sub)
> +               return VLC_EGENERIC;
> +
> +            int64_t* pstart = (int64_t*)va_arg( args, int64_t * );
> +            int64_t* pend = (int64_t*)va_arg( args, int64_t * );
> +
> +            if (!demux_Control (p_sub->p_demux ,
> +                    DEMUX_GET_SUBTITLE_CUR_SENTENCE_TIME +
> +                    i_query - INPUT_GET_CUR_SENTENCE_TIME,
> +                    pstart, pend))
> +                return VLC_SUCCESS;
> +            else
> +                return VLC_EGENERIC;

This assumes that the subtitle demux is re-entrant - which it is not.

Compare INPUT_ADD_SUBTITLE / INPUT_CONTROL_ADD_SUBTITLE,

On Sun, Mar 29, 2015 at 11:42 PM, Goldy Liang <goldyliang at gmail.com> wrote:

>
> Hi Rémi,
>
> Thanks for the response and it is very inspiring. Please see inline
> comments below and my new ideas.
>
> Goldy
>
>
>
>
>> Date: Wed, 25 Mar 2015 19:08:55 +0200
>> From: R?mi Denis-Courmont <remi at remlab.net>
>> To: vlc-devel at videolan.org
>> Subject: Re: [vlc-devel] [PATCH 1/2] Navigation enhancement for
>>         language        learning
>> Message-ID: <14274086.51sHASZh0G at basile.remlab.net>
>> Content-Type: text/plain; charset="utf-8"
>>
>> > On Thu, Mar 19, 2015 at 6:13 AM, Goldy Liang <goldyliang at gmail.com>
>> wrote:
>> > > I just had a glance how EsOutSend works, but I don't think it is good
>> to
>> > > implement there. Looks like EsOutSend would be called by all kinds of
>> > > modules of demuxer or access. In EsOutSend, if I want to exact the
>> time
>> > > for subtitle, I have to check whether it is sent by the subtitle
>> modules.
>>
>> I am not intimate with the ES output, but the type of the track, and its
>> active status should be known there.
>>
>> > > And looks like each subtitle demux will send the data anyway no
>> matter if
>> > > it is selected or default.
>>
>> >From a quick look, the track is active if it has a non-NULL decoder_t
>> pointer
>> - but again, I am not an expert with es_out.c.
>>
>> It does not matter anyway. You can collect the timings for inactive tracks
>> too. That way, seeking will work even if the user just changed the
>> subtitle
>> track. It is only when actually seeking that the input thread need to know
>> which track is active, to query the timings from the correct track.
>>
>
> [Goldy]  If we want to seek a universal solution, yes EsOutSend may work.
> However looks like DecoderProcessSpu could be better. If my understanding
> is right, it should be called by any kinds of subtitle, then a specified
> decoder is called for different types by calling "p_dec->pf_decode_sub".
> (Correct me if I am wrong).
>
> So the idea could be that, the decoder keeps and caches all the decoded
> subtitle timing information on the fly, so that the input thread is able to
> query the decoder module to get the timing of a prev/current sentence. Or
> even the timing of the next sentence (if it is ever decoded). The input
> thread could query the decoder to dump all cached subtitle data as well
> (for future use).
>
> This solution should work for most (if not all) subtitles, even if it has
> the limitation like:
>    - skip to next sentence may not work.
>    - skip to previous sentence may not work either, if the user has just
> done a forward jump (information of prev sentence not known)
>
>
>> > > Even if we can identify the subtitle data and save it, it all depends
>> how
>> > > much data has been sent so as to navigate based on that.
>>
>> Yes. There would be a problem if a sub.itle demuxer sent a single
>> subtitle as
>> multiple blocks. But then again, the ES output works with all file
>> formats and
>> all codecs.
>>
>> It will be very difficult to explain users that the feature only works
>> with some
>> subtitle formats but not all, if we go with the subtitle.c solution.
>>
>
> [Goldy] It is definitely good if the function can work for all subtitle
> formats.
>
> However besides working on the universal solution, I would still want to
> keep the subtitle.c
> solution because the stand-alone text-based subtitle files have the least
> limitation for
> subtitle based navigation, and I believe this kind of subtitle resources
> are very common
> in the internet. So, if there is subtitle loaded by subtitle.c, totally
> perfect, if not, the universal
> solution still works for most functions.
>
>
>> > > For example, if the data of next sentence not sent yet, we can not
>> jump
>> > > forward one sentence; or if the user just changes the slider bar and
>> jump
>> > > to a specific position, we may be missing the time data for the
>> previous
>> > > sentence, etc. And another problem is where to save the time data and
>> how
>> > > the input thread can access it. In a word, it looks complicate and not
>> > > good to me.
>>
>> Jumping forward is not possible as such with the ES output. Still fast
>> forward
>> would be feasible if a little difficult. Also, fast forwarding is usually
>> more
>> robust than seeking.
>>
>
> [Goldy] Doing a fast forward would be a good idea. It looks a bit hard but
> obviously workable.
> Also, if a prev-sentence information is not there, a fast backward should
> also work.
>
>
>>
>> I would however think that seeking to previous and current subtitles is
>> much
>> more essential to the intended use case than seeking to the next subtitle.
>>
>
> [Goldy] When the user wants to review a video clip for language study,
> seeking forward is a
> useful function.
>
> I think I can use the interface I mentioned above to dump the cached
> subtitle information to the interface
> module, then the interface can save it in a standalone subtitle file which
> can be used later on for navigation
> purpose.
>
>
>>
>> > > On the other hand, by controlling the demux, it is more simple and
>> > > provides more flexibility for future development.
>>
>> Controlling the demux is likely simpler if you stick to subtitle.c. But
>> the
>> amount of changes in src/input/ as a whole is comparable if not larger.
>>
>> Still, the big problem with the demux approach is tht it will affect a
>> lot of
>> different demux plugins. On top of my head, besides subtitle, avformat,
>> AVI,
>> dvdnav, dvdread, MKV, MP4, MPEG-PS, MPEG-TS can contain subtitles, and
>> that is
>> not even exhaustive.
>>
>> > > I have more features in my mind about language learning, for example,
>> we
>> > > can show all subtitles in a separate window so that the user can
>> glance
>> > > and jump to any sentence he wants.  This requires the interface
>> module to
>> > > be able to access the whole subtitle data instead of just the time of
>> > > prev/next sentence. I would like to make the subtitle demux to clone
>> and
>> > > dump the whole subtitle data through the input thread control to the
>> > > interface. That would be consistent with the current mechanism that
>> the
>> > > subtitle demux reports the timestamps of prev/cur/next sentences to
>> input
>> > > thread then to interface. If we do that in es_out, I think it would
>> not be
>> > > possible to allow the interface to access the subtitle data freely.
>>
>> The least intrusive way to dump the whole subtitle track for display
>> and/or
>> random access ahead of time in a format-independent fashion is to run the
>> demuxer in a tight loop with a purpose-specific es_out_t instance that
>> just
>> records all the subtitles for later use and discards everything else.
>>
>> An example of custom es_out_t is the Bluray plugin.
>>
>
> [Goldy] Dumping the subtitle data like your said would be a next-step for
> me.
> However dumping the subtitle data loaded in subtitle.c would be more easy
> for
> now.
>
>
>>
>> --
>> R?mi Denis-Courmont
>> http://www.remlab.net/
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20150330/14419637/attachment.html>