Following some interesting scheduler benchmarking [earlier story], a discussion of the results has continued on the lkml. Though the O(1) scheduler offered impressive boosts in most cases, it was noticed that pipe latency increased, and pipe bandwidth decreased [earlier story].

Mike Kravitz suggested, "I believe the decrease in pipe bandwidth is a direct result of the removal of the '__wake_up_sync' support," then asking what the purpose behind this code was in the first place. Andrea Arcangeli explained that the code was an optimization for SMP servers to put processes sharing a pipe on the same CPU if they exceed the alloted buffers. (A full pipe will cause one process to block the other) The tests offered in this thread show that when multiple processes can block the other, keeping them on the same CPU offers a significant gain in bandwidth.

Keith Owens released v2.0 of kuild 2.5 on April 5, boasting a 30% speed improvement over kbuild 2.4 thanks to a rewrite of the core code [earlier story]. He announced that kbuild 2.5 was ready for kernel inclusion May 1 [earlier story], and a debate followed [earlier story]. There has still been no public word from Linus as to when kbuild 2.5 will be merged into the mainline 2.5 development kernel.

Much of the debate is surrounding Keith's choices for "getting it right". In his own words, "kbuild 2.4 defaults to doing a (possibly) inaccurate build after changes," whereas "kbuild 2.5 defaults to always doing an accurate build, no matter what has changed." With the rise of more and more kernel patches, Keith points out that more and more people are going to need the benifits of kbuild 2.5. He concludes, "the default for kernel build must be a safe and accurate build."

kbuild 2.5 is currently at revision 2.4, available here.

Robert Love recently posted a patch (developed by MontaVista) to the lkml bringing the preemptible kernel patch to the MIPS architecture.

Jeremy adds: Robert continues to keep the preempt patch up-to-date with the latest 2.4 kernel and the latest 2.4 -ac patch. His latest patches can be found here

Randy Hron posted some additional O(1) scheduler benchmarks [earlier story], this time illustrating how the scheduler patch seems to decrease pipe bandwidth and increase pipe latency. (neither affect is good)

The thread goes on to talk about Davide Libenzi's Balanced Multi Queue Scheduler patch (BMQS). Andrea Arcangeli says of this scheduler (which he has in his -aa kernel tree), "Davide's patch reduces the complexity of the scheduler from O(N) where N is the number of tasks in the system, to O(N) where N is the number of simultaneous running tasks in the system. It's also a simple optimization and it can make responsiveness even better than the mainline scheduler."

The third scheduler mentioned is the MQ, or Linux MultiQue Scheduler. The Linux Showcase & Conference web site offers an informative comparison of the Default SMP Scheduler (DSS), the MultiQueue Scheduler (MQS), and the Pooled MultiQueue Scheduler (PMQS).

With Keith Owens' announcement that kbuild 2.5 is ready for inclusion into the 2.5 development tree, a discussion on the lkml followed. The big complaint was based on Keith Owens decision to not support modversions (versioned symbols) with this release. Martin Dalecki compared the Linux design process to the Solaris design process, the latter intended to illustrate a better method. Alan Cox explained:

"Now I don't actually give a hoot whether you implement the module binding via /proc/kernel.so and C++ like mangling hacks or the _R stuff we do now but don't confuse the Linux approach of putting a few million users before a few binary module ISV's with the Solaris one."

Kai Germaschewski in turn complained about Keith's style of making such a large change and breaking modversions. Keith explains:

"I have been waiting for somebody to raise the "why not do one bit a time" argument for kbuild. That is exactly what I have done! Modversions are completely broken but are not required for a development kernel, they will be done later. There are 89 'FIXME' comments in the Makefile.in files where changes to source code should be done to clean up the include mess, those changes will be done later."

For full details, read on...

Randy Hron posted some benchmark results comparing numerous different 2.4 kernel branches. He says, "On an OSDL 4 way x86 box the O(1) scheduler effect becomes obvious as the run queue gets large."

Using tbench with 192 processes, his tests show the latest O(1) incarnation to offer approximately a 340% improvement in throughput! (Both Alan Cox's -ac patch and J.A. Magallon's -jam patch set include the latest O(1) scheduler, and both show the dramatic improvement in throughput) Randy intends to do more testing, the results of which will be posted here.

Marcelo released the 2.4.19-pre8 stable Linux kernel today, explaining that -pre9 would follow and then -rc1. 2.4.18 was released on February 25'th, over two months ago. Marcelo explained the delay:

"One might ask why 2.4.19 is taking so long to be released. The reason are the IDE changes: They need to be widely tested, its _really_ critical code."

You can find the full changelog here.

As a follow-up to his announcement of kbuild 2.5 v2.3, Keith Owens announced that kbuild 2.5 (the new kernel-build system) is ready for inclusion into the mainline 2.5 development kernel tree.

Keith writes: "It is faster, better documented, easier to write build rules in, has better install facilities, allows separate source and object trees, can do concurrent builds from the same source tree and is significantly more accurate than the existing kernel build system." He goes on to ask if Linus wishes to completely replace the old kbuild with the new, temporarily breaking unsupported architectures, or to have the two build systems coexist for a few releases.

In September of 2001, Keith Owens modified modutils to detected "tainted" kernels, or kernels with binary-only modules loaded into them. When binary-only modules are loaded and cause problems, kernel developers don't have access to the source code and are hence unable to debug the problem. Knowing whether a kernel is tainted can save them much wasted effort.

The original discussion on this started in this older thread with Alan Cox's release of 2.4.9-ac5. Keith Owens explained his changes in this thread.

When a binary-only module is added to the running kernel, the following message is displayed: "Warning: Loading %s will taint the kernel..." (%s being replaced with the name of the binary only module) As a result of a recent lkml thread, with the release of modutils 2.4.16 the message also offers this URL to the lkml FAQ with a more complete explanation.

A linux server is capable of functioning as an assortment of different network devices. For example, it can act as a router, a switch, or even a network bridge. In the latter case, two or more ethernet networks are linked, or bridged together logically and transparently to make one larger network. Learn more about bridges in this faq.

Torrey Hoffman recently posted information on a patch he's written to the ethernet bridge driver to allow encryption in an ethernet bridge. His driver patch is intended as a tool for further development, currently applied against the 2.4.17-pre7 kernel. The example usage he describes is running multiple MPEG streams across the bridge, encrypting each with a different key to prevent the video from being copied when transferred across the network. Find more information on the project's home page. The patched ethernet bridge driver uses the AES (aka Rijndael) encryption algorithm. For Torrey's announcement email, read on.

Earlier this month Keith Owens announced v2.0 of kbuild 2.5, offering among other improvments a 30% speed increase over kbuild 2.4. (kbuild 2.4 is the currently used build system in the 2.4 and 2.5 kernel trees.) Several days ago, v2.1 of kbuild 2.5 was released, moving towards kernel inclusion.

When asked what he was waiting for, Keith replied, "For me to be satisfied that the code is stable, the rewrite with go faster stripes is less than four weeks old." [Earlier Story]

v2.1 offers a few minor bug fixes. Keith comments, "You know the code is getting stable when most of the changes are documentation and white space from Lindent :)" Later he mentions another improvement the new system has over its predecessor, "With kbuild 2.5 you do not need to 'make clean' before building, unlike the existing build system, kbuild 2.5 gets it right."

The latest IBM middleware for Linux (DB2 Universal Database, WebSphere Application Server, WebSphere Studio Application Developer, and Lotus Domino) is being provided, at no cost, on a 2 CD set, along with Web Services technologies, Linux technical articles, Linux Redbooks, and the very popular Java battle-bots game Robocode. IBM is also providing a worldwide no-cost 2-day Linux workshop, which includes hands-on labs for installing and configuring Linux.

Robert Love recently submitted a group of O(1) scheduler patches for Alan Cox's 2.4-ac branch. The eight patch set brings the ultra-scalable scheduler up to date in the 2.4 stable tree, back porting the many fixes and improvements that were up to now only found in the 2.5 development tree. Robert highlights the addition of the migration thread as the most important change. Though the currently submitted patches are only for 2.4.19-pre7-ac2, Robert noted that he'd be making patches available for 2.4.18 and 2.4.19-pre7 shortly.

Earlier this year, Linus agreed to test out BitKeeper, Larry McVoy's source management tool. Its non-open-source licensing has lead to a fury of protest and discussion.

Erich Focht submitted a patch to the lkml, fixing a bug in migration threads (in the 2.5.8 development kernel) that lead to a deadlock, or frozen system. In the process of fixing the bug, he also worked to cleanup the initialization of the migration threads.

In an earlier email to the lkml, Ingo Molnar explained migration threads:

"The concept is the following: there arenew per-CPU system threads (so-called migration threads) that handle a per-runqueue 'migration queue'. set_cpus_allowed() registers tasks in the target CPU's migration queue, kicks the migrating thread and wakes up the migration thread. The migrating thread unschedules on its source CPU, at which point the migration thread picks the task up and puts it into the local runqueue."

The recent thread is an interesting read, mostly between Erich Focht, Robert Love and William Lee Irwin III.