[vls-devel] Re: Patch to regulate network output
Jean-Paul Saman
jpsaman at wxs.nl
Tue Mar 4 23:43:09 CET 2003
Marian
Thanks for helping out on this issue. I would like you to try changing
some other settings. Please read below:
Marian Durkovic wrote:
> Hi Jean-Paul,
>
>
> yes, kernel's behaviour can make this problem even worse.
> However, I'm doing tests on a machine dedicated to Videolan server, i.e.
> nothing else is blocking the kernel from sending packets out to the network
> card.
>
Did you try the preemtible patch for your 2.4.xx kernel?
Which vls version are you using?
> Basically there are 2 issues (source is MPEG2-PS):
>
> 1) When the input is coming from MPEG2 encoder board, the data is
> split into blocks - since the card only outputs full frame 25 times per
> second. That means you'll end up sending block of data once per 40 msec.
> If the block is I-frame the block could be as big as 100 KB with 4 Mbps rate,
> so in other words, you have a big chunk every 600 msec (GOPsize=15)
>
What is the fluctuation of the bitrate? average 4Mbps, max ?, min ?
> 2) When the input is coming from file, vls uses nanosleep with 20 msec
> resolution on Linux. This means even though PCR is correct, vls sleeps
> for too long and afterwards tries to output all the (already late) data
> in big block.
The cause for this can be the extrapolation in tsstreamer.cpp. Apart
from the kernel overhead before a new scheduling point. Did you try the
preemptible kernel?
> Doing further tests after your comments showed, that problem 2) could
> be partially solved by using clock_nanosleep() with high-res timer in
> tsstreamer.cpp This leads to much better packet distribution over time,
> however, there are still some bigger blocks every 120 msec (I or P frame).
>
I and P frames are bigger, so what you got here is a Variable Bitrate
Video (AFAIK).
What is the fluctuation in bitrate here? avg?, max?, min? Remember a DVB
stream from satellite has an average of 5Mbps and can temporarily go up
to 8 Mbps. This is called a VBR stream.
Are you trying to create a Constant Bitrate (CBR)?
Try changing the setting of TS_IN_ETHER in src/server/output.h to
another value.
#define TS_IN_ETHER 7
// TS packets that fit in an ethernet
Start with:
#define TS_IN_ETHER 1
I try to trigger here that vls will send a packet whenever it is ready.
If correct it should even stop the big chunks every 120ms.
> However, problem 1) is still there in full. In videoinput, the usePCR
> flag is set to false so I've tried to change it - but no luck.
> The result is very strange - at the beginning of capture it seems to slow
> down the traffic a bit, but when running for some time it starts outputting
> whole blocks at 40 msec intervals again.
>
Having usePCR set to false makes the vls act as clock slave to the mpeg
encoder board. For direct connection to encoder boards this should work
alright.
Did you try this with a preemptible kernel?
> With the modified netoutput.cpp, in both cases the packets are being sent
> to network in regular intervals - please see the traces below.
>
> I'll be glad to try to fix the tsstreamer.cpp - however, I'm not quite
> familiar with this and have no tools to monitor/analyze PCRs :-(
>
I'll try to help. Try my patch ;-) I have not tested them !!!
Apply from with vls source-dir with:
patch -p0 < vls_patch_for_marian_durkovic.patch
> Thanks in advance for any help and/or comments.
>
>
> With kind regards,
>
> M.
>
>
> vls streaming from file, standard nanosleep()
> (time is in miliseconds, resolution is 5 ms blocks)
> Packets transmitted in big blocks, once per 20 msec
>
>
> Time: 27200, num: 9
> Time: 27220, num: 12
> Time: 27240, num: 16
> Time: 27260, num: 25
> Time: 27300, num: 13
> Time: 27320, num: 8
> Time: 27340, num: 13
> Time: 27360, num: 15
> Time: 27380, num: 25
> Time: 27420, num: 12
> Time: 27440, num: 8
> Time: 27460, num: 13
> Time: 27480, num: 15
> Time: 27500, num: 16
> Time: 27520, num: 28
> Time: 27525, num: 5
> Time: 27540, num: 13
> Time: 27560, num: 7
> Time: 27580, num: 13
> Time: 27600, num: 15
> Time: 27620, num: 24
> Time: 27660, num: 13
> Time: 27680, num: 7
> Time: 27700, num: 13
>
> vls streaming from file, clock_nanosleep() with high-res timer:
> Looks much better, however still big chunks once per 120 msec
>
> Time: 6060, num: 3
> Time: 6065, num: 25
> Time: 6095, num: 1
> Time: 6100, num: 4
> Time: 6105, num: 4
> Time: 6110, num: 4
> Time: 6115, num: 3
> Time: 6120, num: 1
> Time: 6130, num: 6
> Time: 6135, num: 1
> Time: 6140, num: 4
> Time: 6145, num: 4
> Time: 6150, num: 4
> Time: 6155, num: 4
> Time: 6160, num: 3
> Time: 6165, num: 4
> Time: 6170, num: 4
> Time: 6175, num: 3
> Time: 6180, num: 4
> Time: 6185, num: 25
> Time: 6215, num: 1
> Time: 6220, num: 4
> Time: 6225, num: 4
>
> vls streaming from MPEG2 encoder board:
> Packets transmitted in big blocks, 40 msec apart
>
> Time: 46990, num: 34
> Time: 46995, num: 27 => 61 packets at one moment
> Time: 47030, num: 23
> Time: 47070, num: 24
> Time: 47110, num: 27
> Time: 47150, num: 22
> Time: 47190, num: 27
> Time: 47230, num: 25
> Time: 47270, num: 26
> Time: 47310, num: 1
> Time: 47320, num: 24
> Time: 47350, num: 25
> Time: 47390, num: 25
> Time: 47430, num: 25
> Time: 47470, num: 25
> Time: 47510, num: 27
> Time: 47550, num: 25
> Time: 47590, num: 34
> Time: 47595, num: 19 => 53 packets at one moment
> Time: 47630, num: 23
>
> vls streaming from MPEG2 encoder board with patched netoutput.cpp:
> No more big blocks
>
> Time: 3835, num: 5
> Time: 3840, num: 5
> Time: 3845, num: 6
> Time: 3850, num: 6
> Time: 3855, num: 2
> Time: 3875, num: 5
> Time: 3880, num: 5
> Time: 3885, num: 6
> Time: 3890, num: 6
> Time: 3895, num: 5
> Time: 3900, num: 2
> Time: 3915, num: 4
> Time: 3920, num: 6
> Time: 3925, num: 6
> Time: 3930, num: 5
> Time: 3935, num: 6
> Time: 3940, num: 5
> Time: 3945, num: 6
> Time: 3955, num: 5
> Time: 3960, num: 6
> Time: 3965, num: 1
> Time: 3995, num: 4
>
>
>
>
> On Tue, Mar 04, 2003 at 03:39:22PM +0100, Jean-Paul Saman wrote:
>
>>Marian Durkovic wrote:
>>
>>>Hi all,
>>
>>Hi,
>>
>>Thanks for sending us a patch. It is well written and clean.
>>
>>
>>>
>>> in order to solve the problem with large packet bursts I've modified the
>>>netoutput.cpp to ensure that appropriate gap is inserted after each packet
>>>sent to the network. The gap is calculated from the new config parameter
>>>MaxMbps assuming that 10 Mbps stream needs to send 1000 packets/s
>>>(gap = 1 milisecond). When this parameter is unset, no traffic averaging
>>>is performed.
>>>
>>
>>Unfortunatley I do not think this is the way to resolve that issue. This
>>is more a workaround without solving the *real* problem. Netoutput
>>directly writes the packets (an entire UDP packet, with 7 TS packets) to
>>the network stack. At this moment the UDP packet is handled by the kernel.
>>
>>So when the kernel has time it will send the packet to the network card.
>> On a stock Linux kernel (even with most distributions) this might take
>> 200 ms (worst case). This depends on kernel locks. So when a lock is
>>active nothing other can be done and the packets will have to wait. To
>>solve this issue a preemptive kernel with the lockbreaking patches will
>>be needed. This will give the kernel a IRQ latency of 1.5ms and will
>>solve the 200 ms worst case.
>>
>>VLS does an approximation of the original bitrate through extrapolation
>>upon the PCR's found in the stream. To correctly reproduce the original
>>bitrate VLS should do intrapolation of the PCR's instead of
>>extrapolation. This should solve the bursty behaviour when sending
>>packets. The *offending* code is in /src/server/tsstreamer.cpp,
>>functions AdjustClock(), WaitSendDate(). These should be fixed.
>>
>>
>>> One issue related to this is that the standard nanosleep() on Linux
>>>typically sleeps for minimum 20 miliseconds (!) so in order to have this
>>>functionality, we need the high-resolution-timers compiled into the kernel
>>>- see http://sourceforge.net/projects/high-res-timers/
>>
>>Also VLS uses nanosleep to do timing so it will benefit from the
>>high-resolution-timers too. Also a preemptive kernel with lockbreaking
>>patches is needed (http://sourceforge.net/kpreempt and
>>http://www.tech9.net/rml/linux/)
>>
>>Greetings,
>>Jean-Paul Saman.
>>
>>--
>>This is the vls-devel mailing-list, see http://www.videolan.org/streaming/
>>To unsubscribe, please read http://developers.videolan.org/lists.html
>>If you are in trouble, please contact <postmaster at videolan.org>
>>
>
>
> --------------------------------------------------------------------------
> ---- ----
> ---- Marian Durkovic network manager ----
> ---- ----
> ---- Slovak Technical University Tel: +421 2 524 51 301 ----
> ---- Computer Centre, Nam. Slobody 17 Fax: +421 2 524 94 351 ----
> ---- 812 43 Bratislava, Slovak Republic E-mail: md at bts.sk ----
> ---- ----
> --------------------------------------------------------------------------
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: vls_patch_for_marian_durkovic.patch
Url: http://mailman.videolan.org/pipermail/vls-devel/attachments/20030304/de493af2/attachment.txt
More information about the vls-devel
mailing list