"HP has released AdvFS, a file system that was developed by Digital Equipment Corp and continues to be part of HP's Tru64 operating system," announced Xose Vazquez Perez, offering a link to the re-licensed source code. 2.4 maintainer Willy Tarreau replied favorably, "wow! That's awesome. I discovered it in 1999 and 9 years later, it probably remains the most advanced FS I encountered." HP's Linda Knippers explained:

"In case its not clear, this is a GPLv2 technology release, not an actual port to Linux. We're hoping that the code and documentation will be helpful in the development of new file systems for Linux that will provide similar capabilities, and perhaps used to make tweaks to existing file systems."

Interesting features found in AdvFS include, "simplified file system and storage management; flexible multi-device storage pools shared by multiple file systems, with or without a volume manager; exceptional file system availability (no need to take file systems off-line to expand, shrink or reconfigure; snapshots for consistent backups while applications are on-line; ability to recover deleted files); wide range of performance management tools (fine grain control over file system and file placement within the storage pool; on-line rebalancing of files and free space across the storage pool; on-demand or background file and file system defragmentation); and transaction log management, allowing choices for logging metadata and data asynchronously or synchronously."

In April, 2.4 kernel maintainer Willy Tarreau queried the Linux kernel mailing list regarding how the 2.4 kernel is still being used. He followed up summarizing the responses, suggesting that about 5% of 2.4 users run the kernel on old recycled laptops at home or on PDA's and thin clients, running whatever works with no real need to upgrade. Another 5% of the users are on desktop PCs and monitoring stations, not upgrading because "it works". From there, about 50% of the users run the 2.4 kernel on general purpose servers and update regularly, still running the older kernel due to lack of need for new features and lack of time, and possibly due to failed earlier attempts to upgrade. Another 20% use the 2.4 kernel on application specific servers where reliability is of the highest importance. 10% of the users run the older stable kernel on routers, firewalls, and intrusion detections systems, with 1 to 2 year uptimes and limited updates in a business setting, and shorter uptimes and frequent updates in a personal setting. The final 10% or so use the kernel in embedded systems where stability is again very important, and the build tree may be highly modified, causing at least one major network equipment manufacturer to still be shipping devices with the 2.4.2 kernel. Willy continued:

"Based on that and on the workflow people took the time to explain, I realize that the distinction between -pre and -rc is useless (ding! Linus if you read this, don't beat me). In fact, either people want absolute reliability and they pick one kernel from the stable 2.4.X.Y branch, or they want more recent updates and they simply do their shopping in the -master branch, which is fairly easy thanks to the Gitweb interface. But there are almost no testers in 2.4, just users. That means that I don't have to expect immediate feedback when posting a pre-release. And it has happened several times that I got a build error report several weeks after the release.

"Also, since most people do not update more than 1 - 2 times a year, it's not very useful to have more than 1 - 2 new versions a year, especially since we have the stable release. For this reason, I think I will issue stable releases a bit more often for users to quickly get their fixes, but progressively increase the delay between major releases. Those ones will only be issued with new PCI IDs, major driver updates, compiler support, etc..."

"New year, new kernel: Linux 2.4.36 is finally ready and has been checked long enough to be released. Quite a bunch of bugs, build errors and security issues have been fixed since 2.4.35, but all of those fixes were merged into 2.4.35-stable," 2.4 maintainer Willy Tarreau stated, announcing the latest 2.4 stable Linux kernel. He noted, "I should say that I'm quite satisfied of this dual-branch release model which proves to be very successful at separating quick fixes from changes which require more thorough testing." Willy went on to add:

"Concerning future versions, I have nothing pending in the queue anymore. I will then go on with 2.4.36.X when bug fixes come in, and only open 2.4.37 when I get something which I do not consider suitable for 2.4.36.X."

The previous 2.4.35 stable kernel was released in July of 2007. Source level changes can be viewed through the linux-2.4 gitweb interface.

"For the last release, I stated that I thought the 2.6.22.12 release would be the last one in the 2.6.22.y series. Since then, I've received a number of other patches that would be nice to have in the .22.y tree," explained Greg KH. He continued:

"So, for a while, I'll keep the 2.6.22.y tree open, doing new releases every once in a while as they accumulate. I do this, for no other than the selfish reason that I use it every day on my openSuSE 10.3 boxes as that is the kernel base that release is on :)"

Greg KH and Chris Wright have been maintaining a -stable 2.6.x.y patchset for the 2.6.x and 2.6.(x-1) kernels since March of 2005. 2.4 stable kernel maintainer Willy Tarreau has also maintained patches against the 2.6.20 branch since August of 2007, though noted that he'll switch to maintaining the stable 2.6.22 branch once Greg finishes. Adrian Bunk also continues to maintain a -stable 2.6.16 branch of the Linux kernel.

"I have ported [the] adutux driver for [the] ADU series device list from 2.6 to 2.4," Vitaliy Ivanov announced on the Linux USB development mailing list. 2.4 maintainer Willy Tarreau explained that while the backport looked good, as a rule it was best to not merge new drivers into the stable 2.4 branch of the Linux kernel, "2.4 is currently used by people who already are in 2.4 and cannot/do not want to switch, and by people who are looking for close to zero maintenance. Drivers are often a reason to switch away from 2.4, but not to stay in 2.4."

One of the arguments for merging the driver into the mainline 2.4 tree was so it would then be added to the various distribution kernels. Pete Zaitcev explained that this wasn't the way it worked with enterprise kernels, "at least in case of RHEL, such backports never were automatic. In any case, RHEL 2.1 and 3 do not receive new drivers anymore. We only do bugfixes if something comes up. Realistically speaking, 2.4 kernels are just too old for anyone to use." Willy did agree to consider merging the driver if Vitaliy wanted to step up as the official maintainer for the 2.4 backport.

"In the patch you really remove _a_lot_ of stuff," commented Roman Zippel in his reaction to Ingo Molnar's latest updates to the Completely Fair Scheduler. Roman has been consistently critical of Ingo's efforts, asking questions and criticizing Ingo's feedback. He continued, "you also removed a lot of things I tried to get you to explain them to me. On the one hand I could be happy that these things are gone, as they were the major road block to splitting up my own patch. On the other hand it still leaves me somewhat unsatisfied, as I still don't know what that stuff was good for."

Ingo replied to Roman's technical concerns, and pointed out that he'd been traveling for the recent kernel summit, adding, "I bent backwards trying to somehow get you to cooperate with us (and I still haven't given up on that!) - instead of you disparaging CFS and me frequently :-(". Willy Tarreau took a more critical stance, calling into question Roman's motives. He noted that he had been impressed by Roman's original review of the scheduler, but disappointed as the discussion seemed to degenerate, "it's the way you're trying to prove Ingo is a bastard and that you're a victim. But if we just re-read a few pick-ups of your mails since Aug 1st, its getting pretty obvious that you completely made up this situation." Kyle Moffett added, "I get the impression that Ingo re-implemented some ideas that you had because you refused to do so in a way that was acceptable for the upstream kernel. How exactly is this a bad thing?"

"I've just released Linux 2.4.36-pre1," announced 2.4 maintainer Willy Tarreau. He described a new feature found in the first pre-release:

"In private discussions, Solar Designer proposed to restrict the ability to map the NULL address to CAP_RAW_IO capable processes only. The idea behind this was to prevent 'normal' users from trying to exploit NULL dereferences in the kernel which have not been discovered yet. This is purely a preventive measure."

Willy added that a similar feature exists in the 2.6 kernel, "Chris Wright noted that 2.6 already has a somewhat similar feature brought by Eric Paris, which introduces a sysctl by which the admin can set the lower mappable address." He also noted that the feature can be disabled, "Alan Cox indicated that it was desirable to be able to dynamically disable the feature because some (very) rare programs map the NULL pointer to speed up their walk through linked lists by avoiding NULL pointer checks." Finally he asked for testers, "please report any breakage you would detect when enabling it. We're pretty confident that almost every applications will not see any difference since no normal application maps NULL. But it would be interesting to identify the "special" ones."

Greg KH and Chris Wright have been maintaining a -stable 2.6.x.y patchset for the 2.6.x and 2.6.(x-1) kernels since March of 2005. Thus, with the current stable release being 2.6.22, they maintain -stable patches for 2.6.22 and 2.6.21. 2.4 stable kernel maintainer Willy Tarreau noted the currently high patch rate in each of the 2.6 -stable trees and decided to maintain -stable patches against the 2.6.20 tree until things calm down. Adrian Bunk also continues to maintain a -stable 2.6.16 branch of the Linux kernel. Willy explained about his new 2.6.20 -stable patches:

"I proposed Chris and Greg to continue issuing a few more 2.6.20 releases during the time needed for 2.6.21 and 2.6.22 to show a significant drop in their patch rates, which hopefully will be just a matter of a few releases.

"My goal is *not* to do all the hard work they do, but just to backport from their patches those which are meaningful for 2.6.20. For this reason, 2.6.20 releases will always be slightly late and should not contain patches not merged in more recent releases."

"After 6 months of careful integration and testing, I'm happy to announce availability of Linux 2.4.35," 2.4 maintainer Willy Tarreau announced on the lkml. This is the second stable 2.4 kernel released since Willy became the 2.4 kernel maintainer nearly a year ago in August of 2006. Source level changes can be viewed through the linux-2.4 gitweb interface. Willy added:

"I'm very conscious that 2.4 has mostly left desktop PCs and notebooks, but it's still commonly found on servers, route reflectors or firewalls. For this reason, I'm open to merge the small updates required to maintain such systems running (eg: PCI IDs and such), but I will generally refuse all patches which add support for new desktop or notebook-specific hardware, unless the people present very convincing arguments. Those people generally would better upgrade their systems to 2.6."

The 2.4 stable kernel tree has been maintained by Willy Tarreau for a year, since July of 2006 [story]. When recently asked if the tree had been abandoned, Willy replied, "no it's not abandoned at all! The difficulty with 2.4 is to get user feedback on patches. While in 2.6, there are hundreds or thousands of testers for every release, in 2.4, I have to wait longer after every release in order to start collecting problem reports, or confirmation of fixes. People using it in production generally cannot reboot a machine in the evening just to try a patch. Also, subsystem maintainers have less time to spend on it and are themselves slowed down by the slow feedback process."

As to the upcoming 2.4.35 kernel, Willy noted, "I still have a few patches to merge, and I expect to be able to work on them next week-end, then release -rc1, which I expect to be the only -rc before -final (which will probably not be true as in every -rc)."

2.4 kernel maintainer [story] Willy Tarreau ran some tests to compare Con Kolivas [interview]'s Staircase Deadline CPU scheduler [story] with Ingo Molnar [interview]'s new Completely Fair Scheduler [story]. He summarized his experiences:

"I think that CFS is based on a more promising concept but is less mature and is dangerous right now with certain workloads. SD shows some strange behaviours like not using all CPU available and a little jerkyness, but is more robust and may be the less risky solution for a first step towards a better scheduler in mainline, but it may also probably be the last O(1) scheduler, which may be replaced sometime later when CFS (or any other one) shows at the same time the smoothness of CFS and the robustness of SD."

Some debate was raised by logic added since CFS version 3 to automatically nice the X process for better GUI responsiveness. The CFS changelog comment labels the change as a usibility fix explaining, "automatic renicing of kernel threads such as keventd, OOM tasks and tasks doing privileged hardware access (such as Xorg)." Ingo posted a standalone patch demonstrating how these processes are detected and automatically niced, offering it for inclusion into Con's Staircase scheduler. Willy concurred that it was a good idea, "I think it could be a good idea since you recommend to renice X with SD. Most of the problem users are facing with renicing X is that they need to change their configs or scripts. If the kernel can reliably detect X and handle it differently, why not do it ?" Con was less convinced, "hmm well I have tried my very best to do all the changes without changing 'policy' as much as possible since that trips over so many emotive issues that noone can agree on, and I don't have a strong opinion on this as I thought it would be better for it to be a config option for X in userspace instead. Either way it needs to be turned on/off by admin and doing it by default in the kernel is... not universally accepted as good."

2.4 kernel maintainer Willy Tarreau [story] announced the release of the 2.4.34 stable Linux kernel, "2.4.34 brings the usual bunch of security fixes, bugfixes, and adds support for gcc 4 to x86, x86-64 and sparc64, thanks to Mikael Pettersson's work." Willy also released the 2.4.33.7 kernel with a security fix added in 2.4.34-rc3. He went on to note some caveats:

"One user reported regular panics with aacraid since 2.4.32, so there's no regression here. I will seek for some help to get this fixed in 2.4.35. I also get reports of people getting trapped by NIC vendors who suddenly change their ethernet chips with no big warning notice. The i82546GB chip which replaced the i82546EB in e1000 cards come to mind. It is not supported by the driver in 2.4.34 but I will try to solve this in 2.4.35 (right now, you have to download the vendor's drivers when you replace a NIC). Another driver should get some lifting : skge. I have got a few reports of problems with the vendor's sk98lin driver and I noticed the same problems at work (UDP becoming silent on NFS server)."

Willy Tarreau replaced Marcelo Tosatti [interview] as the 2.4 stable Linux kernel maintainer in August of 2006 [story]. In response to a series of compilation fixes sent to the lkml by Mariusz Kozlowski, Willy suggested that all patches would be postponed until 2.4.34 is released. He suggested that in the interum the appropriate subsystem maintainers should be contacted to determine whether or not each of the patches should be merged, "we would merge the accepted patches and those without any reply which we consider relevant early in the 35-pre cycle so that people have some time to inform us about the potential conflicts they encounter."

Willy went on to describe how to determine who maintains each of the files, "check for the maintainer in the files themselves, or in the MAINTAINERS file. As a rule of thumb, if a file has not changed in the last 3 years and its maintainer is not one of the active ones you regularly see posting on LKML, then there are great chances that the file is unmaintained." He continued, "generally, the core subsystems (network, filesystems, archs, ...) are still well maintained by people who *really want* to validate the patches before forwarding them upstream." He went on to note, "they're all busy, so make the question simple enough so that they can quickly reply with ACK/NACK/QUEUED. Keep me CCed so that you don't have to forward me the response. Generally, they will reply within one week." Willy concluded, "if there's no easily identifiable maintainer anymore, or if some maintainers don't reply, then it becomes my job."

In August 2006 Adrian Bunk took over maintainership of the 2.6.16.y stable kernel [story]. With the release of the 2.6.16.30-pre1 patch, concerns were expressed as to what makes a stable tree. The inclusion of new features led -stable maintainer Greg K-H to observe, "all of these patches seem like these are new features being backported to the 2.6.16.y kernel, which is not really allowed under the current -stable rules." Adrian responded, "they add support for additional hardware to the saa7134 driver. If you look at the actual diff there's not much that could cause any regression since nearly all of these changes don't change anything for the already supported cards." Greg cautioned, "if you want to accept new drivers and backports like this, I think you will find it very hard to determine what to say yes or no to in the future. It's the main problem that everyone who has tried to maintain a stable tree has run into, that is why we set up the -stable rules to be what they are for that very reason."

Willy Tarreau, maintainer of the 2.4 stable kernel [story] joined in with several others expressing concerns about keeping the 2.6.16 tree stable, "when I started the 2.4-hotfix tree nearly two years ago, I wanted to avoid merging driver changes as much as possible. And particularly, I avoided to add support for new hardware. The reason is very simple. I want to be able to guarantee that if 2.4.X works, then any 2.4.X.Y does too so that they can blindly upgrade." Adrian disagreed, "bugfixes causing regressions are much more likely than new hardware support adding regressions." He went on to note that he has two rules for accepting patches, first the patch must be in Linus' tree, and second he directly asks the patch authors and subsystem maintainers for feedback. "I do know that the only value of the 2.6.16 tree lies in a lack of regressions and act accordingly," Adrian added, "but I'm trying to do this in a pragmatic way."