[vlc-devel] a few questions about module behaviour

[Tomas Carnecky]

Laurent Aimar wrote:
> Hi,
> On Tue, Sep 25, 2007, Tomas Carnecky wrote:
>> The video resolution or sound format of the input stream can change at
>> any time. I have taken care of the video problem by making the output ES
>> fixed size and allocating a canvas of that size. When an input frame
>> comes in, it is copied into the canvas, and than the canvas is
>> duplicated into the output ES. That works reasonably well, I'll have to
>> figure out something for the changing sound format (but that's less of
>> an issue right now).
>  You can add and delete ES at the runtime, and so every time the video/audio
> properties change.
>  Then you may use the sout module transcode and gather like this:
>  transcode{vcodec=I420,with=720,height=576,acodec=s16l,samplerate=44100,channels=2}:gather:your_sout_part
>  to be sure to create a video/audio ES that never change.
> (It might be a bit tricky if you have more than one audio or one video)

How does vlc convert the input frames to the predefined width/height?
does it scale them? or copy to the bottom left corner? or center them?

> 
>> It sort-of works if I send out only video frames, but once I add audio
>> vlc goes crazy and complains about bad timing, like:
>> [...] audio drift is too big (236985), dropping buffer
>> [...] late picture skipped (2240)
>>
>> Once thing that really amazed me is when I changed Demux() to only do:
>>   copy = block_Duplicate(canvas)
>>   copy->pts = date_Increment(video.pts, 1)
>>   es_out_send(copy)
>>   return 1;
>> I still saw 'late picture skipped' messages. I don't understand, I can
>> hardly push out frames faster than that, or is there a scheduling
>> problem somewhere?
>  You have to call es_out_Control( p_demux->out, ES_OUT_SET_PCR, pcr )
> regularly (for example every time you call es_out_Send on video data.
>  The value pcr is used to regulate the way VLC eats data. I think the right
> value in your case can simply be the video timestamp.

Does it mean vlc will automatically duplicate/drop frames? Well this
will work with video for sure, but what about audio? If I send an audio
block with pts=0 and 44100 frames (one second of audio data), and the
next block with pts=1100000 (because of a delay when it was captured)
and 44100 frames, I don't want vlc to insert silence, I want vlc to play
it like nothing happened and only pay attention to the block size
(number of frames), and only if the difference between the number of
frames fed to the muxer and the actual time is too big then insert
silence. I can't guarantee that the pts-difference between two blocks
with sound samples matches the number of frames in the first one. Maybe
there is a slightly bigger difference in the pts but the sound samples
are continuous.

btw, do I need to set p_block->i_samples/i_length/i_rate?

>  The value you return on DEMUX_CAN_CONTROL_PACE (p_demux->pf_control) is also
> important :
>  - VLC_TRUE: it means that your source use the same clock than the PC vlc is
> used on (as you use gettimeofday, I think it is the right choice)
>  - VLC_TRUE: your source use a non synchronized clock,
>  es_out_Control( ES_OUT_SET_PCR ) will be use to correct the drift between
> the distant clock and the PC one.

Thanks, I'll add that to my code.

> (And don't forget to return a sensible value on DEMUX_GET_PTS_DELAY, like 300ms)

I do that, but what's the exact meaning of it?

tom