[multicat-devel] Issues with recording and playback

[Christophe Massiot]

Le 16 janv. 2012 à 23:10, Christophe Massiot a écrit :

> I confirm the problem. I will think about it…

I have tracked it down to what (I think) is a kernel bug.

For starters, you should read: http://stackoverflow.com/questions/6834929/linux-virtual-memory-parameters
and familiarize yourself with /proc/sys/vm/dirty_*

The normal behaviour of the kernel is to write back dirty pages in a background task when it reaches a first threshold, or in the process that tries to allocate dirty memory (= direct writeback) when it reaches the second threshold. In the second case write() is blocked, which is what we observe here.

Background writeback is started when the amount of dirty pages amounts to 10 % of the system memory (default value of dirty_background_ratio). In my system with 2 GB of RAM (quite standard nowadays) and a ~3 Mbits/s stream, it is reached only after 568 seconds.

Direct writeback (with a blocking write()) is started at 20 % (dirty_ratio), that is 1136 s in my case.

BUT, there is another mechanism that expires dirty pages after 30 s (default value of dirty_expire_centisecs). The kernel documentation states that dirty_expire_centisecs works with background writeback, but in our experience it starts a direct writeback ! That's the reason why I think it is a kernel bug.

So there are several solutions:

1/ Raise dirty_expire_centisecs above the amount of time needed to start background writeback, in my case 568 s (it may have annoying side-effects)
2/ Lower dirty_background_ratio to be able to start background writeback before 30 s, but in my case 1% (the minimum value) is still not enough!
3/ Put less RAM into your machine :)
4/ Work with streams with a higher bitrate :)
5/ Set dirty background_bytes to something sensible, like 1MB (it overrides dirty_background_ratio)
6/ Fix the kernel bug

The fifth solution is what I'm going for ; it is much better but I am not 100 % sure it works all the time (feedback needed).

The sixth is out of my league, but if you have acquaintances in the kernel world, it may be worth mentioning the problem. Actually it may have gone unnoticed so far because it only happens in circumstances where you have A/ a lot of memory B/ relatively low-bitrate streams (compared to cp or dd) and C/ real-time constraints.