Discussing the latest breakage of the linux-next tree, Stephen Rothwell noted that the problem went unnoticed due to the arm tree not currently being included, "this is why I would have liked you to participate in the linux-next tree ...". Arm maintainer Russell King questioned the usefulness, saying, "linux-next will not give me anything which -mm isn't giving me. As I said in the discussion, linux-next value is _very_ small for me. Sorry but true." Several stepped in to offer some reasons that the linux-next tree is useful.

Andrew Morton noted, "putting arm into linux-next means that Stephen (and git) handle the merges rather than having me (and not-git) do it. Which helps me. I expect that linux-next will get a lot more cross-compilation testing than -mm. Which helps you." Greg KH added, "getting your stuff into linux-next would provide a public place for others to base off of, making it easier for them to send patches to you ensuring that they apply properly. Which in the end, will help others be able to contribute easier, and help you by getting patches you do not need to rebase yourself." Stephen Rothwell summarized the advantages for a maintainer:

"5 times a week your tree gets merged with lots of other code destined for Linus' next release. From this you get to find out about things in other trees that clash with yours. This tree gets built on several architectures for several configs (including arm). So you find out if other trees will break yours. I am happy to build more (basically all) the arm configs as I have offered before."

"This is a formal announcement of Lguest64," Steven Rostedt said in an email posted to the Linux Kernel mailing list. He explained:

"Most are aware of the little puppies (lguest32, or simply lguest, or in some circles "rustyvisor"). But this time the puppies ate a bit too much. No more lean and mean puppies, now we got big fat lazy ones. Running on the hardware that's too lazy to do full virtualization. Yes, lguest now runs on x86_64!"

Steven went on to caution that lguest64 is still a new code base, "lguest64 is still going through a bit of growth pains, but its getting better. It's to a point that we are not that afraid to bring it to the dog show." The list of items left to do include getting SMP working for both the host and the guest, matching Rusty Russel's lguest32 feature set, and greatly optimizing the performance of the code. Steven noted that the goal is to ultimately get the 64-bit version of lguest merged into the mainline kernel.

Some entertaining lguest documentation discussed in an earlier story was merged into the mainline kernel with the commit message, "the netfilter code had very good documentation: the Netfilter Hacking HOWTO. Noone ever read it. So this time I'm trying something different, using a bit of Knuthiness." Both Netfliter and lguest, as well as the documentation for both, were written by Rusty Russell. He describes the lguest driver as, "a simple hypervisor for Linux on Linux. Unlike kvm it doesn't need VT/SVM hardware. Unlike Xen it's simply 'modprobe and go'. Unlike both, it's 5000 lines and self-contained."

Downloading the 2.6.23-rc2 kernel and looking in the "drivers/lguest/" directory I found a simple README that kicks off an interesting documentation process, beginning, "welcome, friend reader, to lguest." It goes on to note, "I can't think of many 5000-line projects which offer both such capability and glimpses of future potential; it is an exciting time to be delving into the source!" At the end of the included README is a hint as to how to find the rest of the documentation, which is embedded inline within all the lguest files. Read on to begin the exploration into lguest and its documentation.

"Lguest is an adventure, with you, the reader, as Hero," began some documentation for lguest recently submitted by Rusty Russell. The documentation continued, "but be warned; this is an arduous journey of several hours or more! And as we know, all true Heroes are driven by a Noble Goal. Thus I offer a Beer (or equivalent) to anyone I meet who has completed this documentation. So get comfortable and keep your wits about you (both quick and humorous). Along your way to the Noble Goal, you will also gain masterly insight into lguest, and hypervisors and x86 virtualization in general."

Andrew Morton noted that he would consider the documentation patches for inclusion in the 2.6.23 kernel, to which Rusty replied, "indeed, no code changes, and I feel strongly that it should go into 2.6.23 because it's *fun*. And (as often complained) there's not enough poetry in the kernel." Linus Torvalds quipped, "there's a reason for that," going on to rhyme, "there once was a lad from Braidwood, with a wife and a hatred for FUD, he hacked kernels for fun, couldn't get them to run, but he always felt that he should." He added, "so when you say 'there's not enough poetry', next time you'll know why. You *really* don't want want poetry." This led to numerous additional poetic submissions about which Rusty noted, "there was a poetic infection, which distorted the kernel's direction, the code got no time, as they all tried to rhyme, and it shipped needing lots of correction."

Rusty Russell's lguest was recently merged into the upcoming 2.6.23 Linux kernel. The merge comment describes the project, "lguest is a simple hypervisor for Linux on Linux. Unlike kvm it doesn't need VT/SVM hardware. Unlike Xen it's simply 'modprobe and go'. Unlike both, it's 5000 lines and self-contained." The comment goes on to note:

"Performance is ok, but not great (-30% on kernel compile). But given its hackability, I expect this to improve, along with the paravirt_ops code which it supplies a complete example for. There's also a 64-bit version being worked on and other craziness.

"But most of all, lguest is awesome fun! Too much of the kernel is a big ball of hair. lguest is simple enough to dive into and hack, plus has some warts which scream 'fork me!'."

Andrew Morton [interview] sent out the latest lguest patches for review, noting that he intends to merge the code into the mainline kernel, "some concern was expressed over the lguest review status, so I shall send the patches out again for people to review, to test, to make observations about the author's personal appearance, etc. I'll plan on sending these patches off to Linus in a week's time, assuming all goes well." The project's FAQ notes, "lguest is designed to be simple to use and modify, with the aim of keeping the codebase small. Currently it's around 5000 lines including userspace utility, whereas kvm is over 10 times that size, and Xen is around 10 times bigger again (of course, both have far more features)."

The lguest patches are written and maintained by Rusty Russell [interview] who also authored Rusty's Remarkably Unreliable Guide to Lguest, the project's documentation. The guide explains, "lguest is designed to be a minimal hypervisor for the Linux kernel, for Linux developers and users to experiment with virtualization with the minimum of complexity. Nonetheless, it should have sufficient features to make it useful for specific tasks, and, of course, you are encouraged to fork and enhance it." In the FAQ, lguest is compared to kvm [story], "kvm requires hardware virtualization support (most recent Intel and AMD chips have it), but it can run almost any Operating System (since it does full virtualization. It also has 64-bit support. Lguest doesn't do full virtualization: it only runs a Linux kernel with lguest support." The FAQ also compares lguest to Xen, "Xen is similar, in that it doesn't need hardware virtualization support (although it can use it), but Xen supports an extensive range of features such as PAE (ie. lots of memory), SMP guests, 64-bit. You have to boot your kernel under the Xen hypervisor; you can't simply modprobe when you want to create a guest."

Michael Marineau announced an updated version of the Linux Kernel Graphing Project, now known as the Free Code Graphing Project. The original code was written by Rusty Russell [interview] and used to generate "a graph of all the .c files in the 2.4.0 Linux kernel: each function graphically represented and named, resulting in 180MB of PostScript."

Michael has long since taken over the project, recently releasing the first new version in a few years explaining, "over the past few months I've slowly gotten the old code base updated to work cleanly on 2.6 kernels as well as being usable on kernels as old as 1.2 :-)" He goes on to note that if you desire a complete print out of the current kernel code as a poster, "it needs to be at least 6ft wide to have a chance at reading the file and function names. Also a high resolution is required; 600dpi is just barely usable." An online version of the new poster can be navigated here.

Rusty Russell [interview] recently posted an updated version of his "Unreliable Guide To Locking". The introduction begins:

"Welcome, to Rusty's Remarkably Unreliable Guide to Kernel Locking issues. This document describes the locking systems in the Linux Kernel in 2.6. With the wide availability of HyperThreading, and preemption in the Linux Kernel, everyone hacking on the kernel needs to know the fundamentals of concurrency and locking for SMP. "

Rusty's excellent guide is quite informative, helping the reader to grasp concurrency, explaining the common types of locks, providing useful examples, listing common problems, discussing locking speed, and much more. All in all, it's an essential reference.

Rusty Russell is a Linux kernel hacker living in Australia, working for IBM's Linux Technology Center. He's also a frequent and talented speaker at Linux gatherings. When talking about Rusty in an earlier interview, Andrew Morton summarized, "he's just a really sharp and witty guy - anyone who has attended one of his sessions at a conference will attest to that!"

Well known for his packet filtering efforts, having written both ipchains and netfilter/iptables, he has continued to make an impressive number of contributions to Linux kernel development. A large sampling of his current projects have been merged into the upcoming 2.6 kernel, including futexes, per-cpu counters, hot pluggable CPU support, and a complete rewrite of the in-kernel module loading code.

For a humorous sample of Rusty's wit, one only needs to look to his email signature which reads, "Anyone who quotes me in their sig is an idiot. -- Rusty Russell." Read on for the full interview.

Mark Wong posted a series of benchmark results from Rusty Russell's Hackbench. Rusty describes Hackbench as a minimized 'chat benchmark' that doesn't use threads or semaphores. The benchmark launches groups of processes that each listen on a given socket, and complimentary groups of processes that write 100 messages to each of the listening sockets, measuring the time this takes. This process is repeated multiple times with an increasing number of groups of processes, therby measuring the scalability of the scheduler with an increasing number of processes.

Mark's results begin with the 2.5.28 development kernel and continue up through the current 2.6.0-test5 kernel. In a second email he also offers results of the -mm tree, beginning with 2.5.66-mm1 and continuing up through 2.6.0-test5-mm2. Andrew Morton [interview] glanced at the results and commented that they looked "great, but tragically incomprehensible", going on to ask for an explanation, "do we rock or do we suck?". Mark replied, "the general trend in the metric indicates everything has been improving, so I think we rock."

Rusty Russell's new module loader was recently merged into Linus' 2.5 kernel tree [story]. This new implementation aims to cleanup and reduce the amount of code in the kernel and user space required to load a kernel module. Additionally, it now removes the requirement that kernel and user space code for modutils have to be in sync.