I have a mutlti-threading program. It can receive/respond packets on the network. When I do a stress test on it, it can process 60 request/response packets per seconds, in kernel 2.6.17, which uses the O(1) scheduler. But when I do the same test in 2.6.23, with CFS scheduler, its performance will drop down to less than 10 request/response packets per seconds. During the test most of the other processes are sleeping in the system. That means no one is competing the CPU with it. But if you use "top", you will see those threads of my program are taking nearly 0% CPU. And if you use "ps", you will always see them in S state. I don't understand why CFS does not give CPU resource to my program. Then I ran some other test and got some more interesting results. That is with the "/proc/sys/kernel/sched_compat_yield" turned on, it will become a lot faster when system is idle (this is reasonable). But as soon as there is a CPU bound process (like a infinite loop program) comes in, it will take out 90% of CPU and my program will become very, very slow. This violate the game rule of CFS since CFS' goal is about completely fair. No process can take out most of the CPU time if there is other processes in need of CPU at the same time. But then if "/proc/sys/kernel/sched_compat_yield" is turned off(this is default), no matter you put in how many CPU bound processes, they will share the CPU time fairly, so my program will not become unreasonable slow. But the problem will then become, if system is idle, my program will be slow, slower than when system is loaded with other processes. I really don't understand why it behaves like this under CFS. Does anyone have any clue to help me out? Thanks.
You should at least try
You should at least try kernel 2.6.24 before, as many CFS bugs were fixed after 2.6.23 (and many others in 2.6.25). If you still get the same behavior, you should report (with appropriate code samples and detailed informations about your program) to lkml.
Bye,
Francesco
The bug fix for CFS after 2.6.24
Francesco,
Thanks very much for your advice. Before I tried 2.6.24 or 25, I would like to understand roughly what kind of bugs of CFS are fixed in them, since porting the new version of kernel to target board will cost quite amount of time and this is what I am short of right now :( Do you know where I can find a related bug fix report to look at?
Thanks,
Bob
Re: The bug fix for CFS after 2.6.24
Scheduler fixes are usually merged from this git tree:
http://git.kernel.org/?p=linux/kernel/git/mingo/linux-2.6-sched-fixes.gi...
You can even look up for scheuler commits on the mainstream tree (using "sched", "scheduler", "cfs", etc. keywords):
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=tags
Unfortunately, the gitweb interface is limited. If you are experienced with git, you can use the command line tools to query changesets on specific files.