Linux: Performance Testing The Kernel

Submitted by Jeremy
on July 15, 2005 - 8:31am

Kenneth Chen announced the Linux Kernel Performance Project, "we decided to run a large set of benchmarks covering core components of the Linux kernel (virtual memory management, I/O subsystem, process scheduler, file system, network, device driver, etc) on a regular basis." Results from 13 benchmark utilities are displayed in both table and graph format, run on several different servers. The 2.6.9 kernel [story] is used as the baseline, compared against various more recent kernels up to 2.6.13-rc1. Ken explains:

"Our goal is to work with the Linux community to further enhance the performance of the Linux kernel. The data available on the site allows community members to closely track performance gains and losses with every version of the kernel. Ultimately, we hope that this data will result in performance increases in Linux kernel development."


From: "Chen, Kenneth W" [email blocked]
To: [email blocked]
Subject: [announce] linux kernel performance project launch at sourceforge.net
Date:	Thu, 14 Jul 2005 13:21:25 -0700

I'm pleased to announce that we have established a linux kernel
performance project, hosted at sourceforge.net:

http://kernel-perf.sourceforge.net

As much as discussed various time in the past on LKML that Linux
kernel needs a systematic and disciplined way to measure and track
kernel performance on a regular basis.  To do that, we decided to
run a large set of benchmarks covering core components of the Linux
kernel (virtual memory management, I/O subsystem, process scheduler,
file system, network, device driver, etc) on a regular basis.
Benchmarks are run on a variety of platforms (4P Intel Xeon processor,
2P Xeon, several ia64 server boxes etc) every week, measuring the
latest snapshot of Linus' git development tree. Comprehensive performance
data from our tests will be published for easy access.

Our goal is to work with the Linux community to further enhance the
performance of the Linux kernel. The data available on the site allows
community members to closely track performance gains and losses with
every version of the kernel. Ultimately, we hope that this data will
result in performance increases in Linux kernel development.

The benchmark result pages are populated with a few benchmarks at the
moment.  In the coming weeks, we will be populating more benchmark data.
Happy surfing and hacking!!


		Ken Chen [email blocked]
		Intel Open Source Technology Center


From: Andi Kleen [email blocked] Subject: Re: [announce] linux kernel performance project launch at sourceforge.net Date: 15 Jul 2005 00:18:03 +0200 "Chen, Kenneth W" [email blocked] writes: > I'm pleased to announce that we have established a linux kernel > performance project, hosted at sourceforge.net: > > http://kernel-perf.sourceforge.net That's very cool. Thanks a lot. Would it be possible to add 2.4.30 numbers and perhaps one or two distro kernels (let's say RHEL3/4, SLES8/9) to the graphs as data points for comparison? These are all very tuned kernels and would show where mainline is worse than them. Also how did you run netperf? Locally or to some other machine? Perhaps that should be documented. Some oprofile listings from a few of the test runs would be also nice. -Andi
From: "Chen, Kenneth W" [email blocked] Subject: RE: [announce] linux kernel performance project launch at sourceforge.net Date: Thu, 14 Jul 2005 15:24:45 -0700 [email blocked] wrote on Thursday, July 14, 2005 3:18 PM > "Chen, Kenneth W" writes: > > I'm pleased to announce that we have established a linux kernel > > performance project, hosted at sourceforge.net: > > > > http://kernel-perf.sourceforge.net > > That's very cool. Thanks a lot. Thank you. > Would it be possible to add 2.4.30 numbers and perhaps one or two > distro kernels (let's say RHEL3/4, SLES8/9) to the graphs > as data points for comparison? These are all very tuned > kernels and would show where mainline is worse than them. We did have a distro kernel in the graph originally and later decided to go with pure mainline kernels for consistency. We will see what we can do to add them in the future. > Also how did you run netperf? Locally or to some other machine? > Perhaps that should be documented. Yes, that was netperf, running locally. Thanks for the suggestion, I will document that appropriately. > Some oprofile listings from a few of the test runs would be also nice. That is in the works. We will upload profile data. I'm having problem with oprofile on some versions of kernel and that is being investigated right now. - Ken
From: Andi Kleen [email blocked] Subject: Re: [announce] linux kernel performance project launch at sourceforge.net Date: Fri, 15 Jul 2005 00:27:48 +0200 > > Some oprofile listings from a few of the test runs would be also nice. > > That is in the works. We will upload profile data. I'm having problem > with oprofile on some versions of kernel and that is being investigated > right now. If you run statically compiled kernels you could as well use the old style readprofile. It just doesn't work with modules. -Andi

Related Links:

Didn't Linus want something l

Anonymous_guy (not verified)
on
July 16, 2005 - 12:18am

Didn't Linus want something like this to track preformance changes? I'm glad someone has finally done this. Good job guys at Intel. =)

cool - should be nice to see

on
July 16, 2005 - 7:34am

cool - should be nice to see some amd64 benchmark results soon.

oh wait ..

duplicate work

Anonymous_guy (not verified)
on
July 18, 2005 - 3:42am

it looks similar to yet another project :
http://ftp.kernel.org/pub/linux/kernel/people/mbligh/abat/perf/perf_matr...

except that the latter runs across more plateforms while the former includes more tests and results are more comprehensive (shown in percentage instead of pure results)

duplicate work

Anonymous_guy (not verified)
on
July 18, 2005 - 3:47am

it looks similar to yet another project :
http://ftp.kernel.org/pub/linux/kernel/people/mbligh/abat/perf/perf_matr...

except that the latter runs across more plateforms while the former includes more tests and results are more comprehensive (shown in percentage instead of pure results)

Duplicate work indeed! ;-)

-bertox (not verified)
on
July 18, 2005 - 5:56pm

Duplicate work indeed! ;-)

What about adding {Free,Net,DragonFly}BSDs + Darwin in the mix?

Anonymous/00006 (not verified)
on
July 19, 2005 - 5:29am

It would be interesting to see how well systems perform, compared to each other.

I think that would be intesting for many people.

People then would be able to judge what *nix will fit them better.

Probably there are other Unix-like OSs which can use good performance profiling, in parallel to development.

What about error bars ?

Ionut (not verified)
on
July 19, 2005 - 8:10am

The deviations are so small ... did you try to compute the standard deviations from several runs ?

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.