Kenneth Chen announced the Linux Kernel Performance Project, "we decided to run a large set of benchmarks covering core components of the Linux kernel (virtual memory management, I/O subsystem, process scheduler, file system, network, device driver, etc) on a regular basis." Results from 13 benchmark utilities are displayed in both table and graph format, run on several different servers. The 2.6.9 kernel [story] is used as the baseline, compared against various more recent kernels up to 2.6.13-rc1. Ken explains:
"Our goal is to work with the Linux community to further enhance the performance of the Linux kernel. The data available on the site allows community members to closely track performance gains and losses with every version of the kernel. Ultimately, we hope that this data will result in performance increases in Linux kernel development."
From: "Chen, Kenneth W" [email blocked] To: [email blocked] Subject: [announce] linux kernel performance project launch at sourceforge.net Date: Thu, 14 Jul 2005 13:21:25 -0700 I'm pleased to announce that we have established a linux kernel performance project, hosted at sourceforge.net: http://kernel-perf.sourceforge.net As much as discussed various time in the past on LKML that Linux kernel needs a systematic and disciplined way to measure and track kernel performance on a regular basis. To do that, we decided to run a large set of benchmarks covering core components of the Linux kernel (virtual memory management, I/O subsystem, process scheduler, file system, network, device driver, etc) on a regular basis. Benchmarks are run on a variety of platforms (4P Intel Xeon processor, 2P Xeon, several ia64 server boxes etc) every week, measuring the latest snapshot of Linus' git development tree. Comprehensive performance data from our tests will be published for easy access. Our goal is to work with the Linux community to further enhance the performance of the Linux kernel. The data available on the site allows community members to closely track performance gains and losses with every version of the kernel. Ultimately, we hope that this data will result in performance increases in Linux kernel development. The benchmark result pages are populated with a few benchmarks at the moment. In the coming weeks, we will be populating more benchmark data. Happy surfing and hacking!! Ken Chen [email blocked] Intel Open Source Technology Center
From: Andi Kleen [email blocked] Subject: Re: [announce] linux kernel performance project launch at sourceforge.net Date: 15 Jul 2005 00:18:03 +0200 "Chen, Kenneth W" [email blocked] writes: > I'm pleased to announce that we have established a linux kernel > performance project, hosted at sourceforge.net: > > http://kernel-perf.sourceforge.net That's very cool. Thanks a lot. Would it be possible to add 2.4.30 numbers and perhaps one or two distro kernels (let's say RHEL3/4, SLES8/9) to the graphs as data points for comparison? These are all very tuned kernels and would show where mainline is worse than them. Also how did you run netperf? Locally or to some other machine? Perhaps that should be documented. Some oprofile listings from a few of the test runs would be also nice. -Andi
From: "Chen, Kenneth W" [email blocked] Subject: RE: [announce] linux kernel performance project launch at sourceforge.net Date: Thu, 14 Jul 2005 15:24:45 -0700 [email blocked] wrote on Thursday, July 14, 2005 3:18 PM > "Chen, Kenneth W" writes: > > I'm pleased to announce that we have established a linux kernel > > performance project, hosted at sourceforge.net: > > > > http://kernel-perf.sourceforge.net > > That's very cool. Thanks a lot. Thank you. > Would it be possible to add 2.4.30 numbers and perhaps one or two > distro kernels (let's say RHEL3/4, SLES8/9) to the graphs > as data points for comparison? These are all very tuned > kernels and would show where mainline is worse than them. We did have a distro kernel in the graph originally and later decided to go with pure mainline kernels for consistency. We will see what we can do to add them in the future. > Also how did you run netperf? Locally or to some other machine? > Perhaps that should be documented. Yes, that was netperf, running locally. Thanks for the suggestion, I will document that appropriately. > Some oprofile listings from a few of the test runs would be also nice. That is in the works. We will upload profile data. I'm having problem with oprofile on some versions of kernel and that is being investigated right now. - Ken
From: Andi Kleen [email blocked] Subject: Re: [announce] linux kernel performance project launch at sourceforge.net Date: Fri, 15 Jul 2005 00:27:48 +0200 > > Some oprofile listings from a few of the test runs would be also nice. > > That is in the works. We will upload profile data. I'm having problem > with oprofile on some versions of kernel and that is being investigated > right now. If you run statically compiled kernels you could as well use the old style readprofile. It just doesn't work with modules. -Andi
Didn't Linus want something l
Didn't Linus want something like this to track preformance changes? I'm glad someone has finally done this. Good job guys at Intel. =)
cool - should be nice to see
cool - should be nice to see some amd64 benchmark results soon.
oh wait ..
duplicate work
it looks similar to yet another project :
http://ftp.kernel.org/pub/linux/kernel/people/mbligh/abat/perf/perf_matr...
except that the latter runs across more plateforms while the former includes more tests and results are more comprehensive (shown in percentage instead of pure results)
duplicate work
it looks similar to yet another project :
http://ftp.kernel.org/pub/linux/kernel/people/mbligh/abat/perf/perf_matr...
except that the latter runs across more plateforms while the former includes more tests and results are more comprehensive (shown in percentage instead of pure results)
Duplicate work indeed! ;-)
Duplicate work indeed! ;-)
What about adding {Free,Net,DragonFly}BSDs + Darwin in the mix?
It would be interesting to see how well systems perform, compared to each other.
I think that would be intesting for many people.
People then would be able to judge what *nix will fit them better.
Probably there are other Unix-like OSs which can use good performance profiling, in parallel to development.
What about error bars ?
The deviations are so small ... did you try to compute the standard deviations from several runs ?