[vls-devel] Re: Patch to regulate network output
Marian Durkovic
md at bts.sk
Tue Mar 4 17:56:23 CET 2003
Hi Jean-Paul,
yes, kernel's behaviour can make this problem even worse.
However, I'm doing tests on a machine dedicated to Videolan server, i.e.
nothing else is blocking the kernel from sending packets out to the network
card.
Basically there are 2 issues (source is MPEG2-PS):
1) When the input is coming from MPEG2 encoder board, the data is
split into blocks - since the card only outputs full frame 25 times per
second. That means you'll end up sending block of data once per 40 msec.
If the block is I-frame the block could be as big as 100 KB with 4 Mbps rate,
so in other words, you have a big chunk every 600 msec (GOPsize=15)
2) When the input is coming from file, vls uses nanosleep with 20 msec
resolution on Linux. This means even though PCR is correct, vls sleeps
for too long and afterwards tries to output all the (already late) data
in big block.
Doing further tests after your comments showed, that problem 2) could
be partially solved by using clock_nanosleep() with high-res timer in
tsstreamer.cpp This leads to much better packet distribution over time,
however, there are still some bigger blocks every 120 msec (I or P frame).
However, problem 1) is still there in full. In videoinput, the usePCR
flag is set to false so I've tried to change it - but no luck.
The result is very strange - at the beginning of capture it seems to slow
down the traffic a bit, but when running for some time it starts outputting
whole blocks at 40 msec intervals again.
With the modified netoutput.cpp, in both cases the packets are being sent
to network in regular intervals - please see the traces below.
I'll be glad to try to fix the tsstreamer.cpp - however, I'm not quite
familiar with this and have no tools to monitor/analyze PCRs :-(
Thanks in advance for any help and/or comments.
With kind regards,
M.
vls streaming from file, standard nanosleep()
(time is in miliseconds, resolution is 5 ms blocks)
Packets transmitted in big blocks, once per 20 msec
Time: 27200, num: 9
Time: 27220, num: 12
Time: 27240, num: 16
Time: 27260, num: 25
Time: 27300, num: 13
Time: 27320, num: 8
Time: 27340, num: 13
Time: 27360, num: 15
Time: 27380, num: 25
Time: 27420, num: 12
Time: 27440, num: 8
Time: 27460, num: 13
Time: 27480, num: 15
Time: 27500, num: 16
Time: 27520, num: 28
Time: 27525, num: 5
Time: 27540, num: 13
Time: 27560, num: 7
Time: 27580, num: 13
Time: 27600, num: 15
Time: 27620, num: 24
Time: 27660, num: 13
Time: 27680, num: 7
Time: 27700, num: 13
vls streaming from file, clock_nanosleep() with high-res timer:
Looks much better, however still big chunks once per 120 msec
Time: 6060, num: 3
Time: 6065, num: 25
Time: 6095, num: 1
Time: 6100, num: 4
Time: 6105, num: 4
Time: 6110, num: 4
Time: 6115, num: 3
Time: 6120, num: 1
Time: 6130, num: 6
Time: 6135, num: 1
Time: 6140, num: 4
Time: 6145, num: 4
Time: 6150, num: 4
Time: 6155, num: 4
Time: 6160, num: 3
Time: 6165, num: 4
Time: 6170, num: 4
Time: 6175, num: 3
Time: 6180, num: 4
Time: 6185, num: 25
Time: 6215, num: 1
Time: 6220, num: 4
Time: 6225, num: 4
vls streaming from MPEG2 encoder board:
Packets transmitted in big blocks, 40 msec apart
Time: 46990, num: 34
Time: 46995, num: 27 => 61 packets at one moment
Time: 47030, num: 23
Time: 47070, num: 24
Time: 47110, num: 27
Time: 47150, num: 22
Time: 47190, num: 27
Time: 47230, num: 25
Time: 47270, num: 26
Time: 47310, num: 1
Time: 47320, num: 24
Time: 47350, num: 25
Time: 47390, num: 25
Time: 47430, num: 25
Time: 47470, num: 25
Time: 47510, num: 27
Time: 47550, num: 25
Time: 47590, num: 34
Time: 47595, num: 19 => 53 packets at one moment
Time: 47630, num: 23
vls streaming from MPEG2 encoder board with patched netoutput.cpp:
No more big blocks
Time: 3835, num: 5
Time: 3840, num: 5
Time: 3845, num: 6
Time: 3850, num: 6
Time: 3855, num: 2
Time: 3875, num: 5
Time: 3880, num: 5
Time: 3885, num: 6
Time: 3890, num: 6
Time: 3895, num: 5
Time: 3900, num: 2
Time: 3915, num: 4
Time: 3920, num: 6
Time: 3925, num: 6
Time: 3930, num: 5
Time: 3935, num: 6
Time: 3940, num: 5
Time: 3945, num: 6
Time: 3955, num: 5
Time: 3960, num: 6
Time: 3965, num: 1
Time: 3995, num: 4
On Tue, Mar 04, 2003 at 03:39:22PM +0100, Jean-Paul Saman wrote:
> Marian Durkovic wrote:
> > Hi all,
> Hi,
>
> Thanks for sending us a patch. It is well written and clean.
>
> >
> >
> > in order to solve the problem with large packet bursts I've modified the
> > netoutput.cpp to ensure that appropriate gap is inserted after each packet
> > sent to the network. The gap is calculated from the new config parameter
> > MaxMbps assuming that 10 Mbps stream needs to send 1000 packets/s
> > (gap = 1 milisecond). When this parameter is unset, no traffic averaging
> > is performed.
> >
> Unfortunatley I do not think this is the way to resolve that issue. This
> is more a workaround without solving the *real* problem. Netoutput
> directly writes the packets (an entire UDP packet, with 7 TS packets) to
> the network stack. At this moment the UDP packet is handled by the kernel.
>
> So when the kernel has time it will send the packet to the network card.
> On a stock Linux kernel (even with most distributions) this might take
> 200 ms (worst case). This depends on kernel locks. So when a lock is
> active nothing other can be done and the packets will have to wait. To
> solve this issue a preemptive kernel with the lockbreaking patches will
> be needed. This will give the kernel a IRQ latency of 1.5ms and will
> solve the 200 ms worst case.
>
> VLS does an approximation of the original bitrate through extrapolation
> upon the PCR's found in the stream. To correctly reproduce the original
> bitrate VLS should do intrapolation of the PCR's instead of
> extrapolation. This should solve the bursty behaviour when sending
> packets. The *offending* code is in /src/server/tsstreamer.cpp,
> functions AdjustClock(), WaitSendDate(). These should be fixed.
>
> > One issue related to this is that the standard nanosleep() on Linux
> > typically sleeps for minimum 20 miliseconds (!) so in order to have this
> > functionality, we need the high-resolution-timers compiled into the kernel
> > - see http://sourceforge.net/projects/high-res-timers/
>
> Also VLS uses nanosleep to do timing so it will benefit from the
> high-resolution-timers too. Also a preemptive kernel with lockbreaking
> patches is needed (http://sourceforge.net/kpreempt and
> http://www.tech9.net/rml/linux/)
>
> Greetings,
> Jean-Paul Saman.
>
> --
> This is the vls-devel mailing-list, see http://www.videolan.org/streaming/
> To unsubscribe, please read http://developers.videolan.org/lists.html
> If you are in trouble, please contact <postmaster at videolan.org>
>
--------------------------------------------------------------------------
---- ----
---- Marian Durkovic network manager ----
---- ----
---- Slovak Technical University Tel: +421 2 524 51 301 ----
---- Computer Centre, Nam. Slobody 17 Fax: +421 2 524 94 351 ----
---- 812 43 Bratislava, Slovak Republic E-mail: md at bts.sk ----
---- ----
--------------------------------------------------------------------------
--
This is the vls-devel mailing-list, see http://www.videolan.org/streaming/
To unsubscribe, please read http://developers.videolan.org/lists.html
If you are in trouble, please contact <postmaster at videolan.org>
More information about the vls-devel
mailing list