"Some people seem to be using 'Acked-by' to mean, 'seems good to me', without necessarily doing a full review of the patch, and instead of trying to change the meaning of 'Acked-by', [the plan is] to have a new sign off which is a bit more explicitly about what it means," Theodore Tso explained in a recent thread on the Linux Kernel mailing list. He continued:

"This was proposed by Andrew and discussed at the Kernel Summit; the basic idea is that it is a formal indication that the person has done a *full* review of the patch (a few random comments from the local whitespace police don't count), and is willing to vouch that the patch is correct, safe, extremely unlikely to cause regressions, etc. If the patch does need to be reverted or fixed because it was buggy, then both the original submitter and the reviewer would bear responsibility and subsystem maintainers might take that into account when assessing the reputations of the submitter and reviewer in the future when deciding whether or not to accept a patch."

Andrew Morton noted that the idea isn't fully fleshed out yet, "we will start introducing Reviewed-by: (I haven't yet quite worked out how yet) but it will be a quite formal thing and it would be something which the reviewer explicitly provided. For now, let's please stick with acked-by". Theodore added, "there was also some discussion about whether or not patches would not be accepted at all without a Reviewed-by, but that probably won't happen initially. The general consensus was to gently ease into it and see how well it works first."

Greg KH and Chris Wright have been maintaining a -stable 2.6.x.y patchset for the 2.6.x and 2.6.(x-1) kernels since March of 2005. Thus, with the current stable release being 2.6.22, they maintain -stable patches for 2.6.22 and 2.6.21. 2.4 stable kernel maintainer Willy Tarreau noted the currently high patch rate in each of the 2.6 -stable trees and decided to maintain -stable patches against the 2.6.20 tree until things calm down. Adrian Bunk also continues to maintain a -stable 2.6.16 branch of the Linux kernel. Willy explained about his new 2.6.20 -stable patches:

"I proposed Chris and Greg to continue issuing a few more 2.6.20 releases during the time needed for 2.6.21 and 2.6.22 to show a significant drop in their patch rates, which hopefully will be just a matter of a few releases.

"My goal is *not* to do all the hard work they do, but just to backport from their patches those which are meaningful for 2.6.20. For this reason, 2.6.20 releases will always be slightly late and should not contain patches not merged in more recent releases."

Following a recent merge request, Linus Torvalds stressed that he was serious about not wanting to merge any big changes after the merge window closes, "get the changes in before -rc1, or just *wait*. If they aren't ready before the merge window opens, they simply shouldn't be merged at all." Jeff Garzik reiterated, "once -rc1 is out there, that means the focus should be on stabilizing the existing codebase. Pushing a big driver update means that effort must restart from scratch. We just don't want to go down that road, which a big reason for the merge window in general." Further when it was noted that the recent changes were heavily tested by the vendor, Jeff stressed the importance of community testing:

"Take a lesson from when I was on Linus's shit-list... twice: Twice, Intel submitted an e1000 update after the merge window closed. Twice, they claimed the driver passed their quite-exhaustive internal testing. And twice, the most popular network driver broke for large masses of users because I took a hardware vendor's word on testing rather than rely on the testing PROVEN to flush out problems: public linux kernel testing.

"I'm not singling out Intel, there are plenty of other hardware vendors that repeat the exact same pattern."

In response to another merge request, Andrew Morton retorted, "argh. I have a backlog of maybe 300 patches here which I am cheerfully ignoring while concentrating on preventing 2.6.23 from being less of a disaster than it has already been." He noted that he was not planning to merge any new code into his -mm tree for 2.6.23 inclusion, "the door for new 2.6.23 material shut two weeks ago. Here, at least." He went on to note:

"Please, stop writing patches. Maybe do something to help get 2.6.23 off its back. Like, go review some of the code which people are cheerfully merging five minutes after having written it."

Recent merges into the upcoming 2.6.23 kernel can be found by browsing the gitweb interface to Linus' 2.6 kernel tree. The 2.6.23-rc1 kernel should be released on or shortly after Sunday the 22nd, two weeks after 2.6.22 was released, and at which time the merge window is closed.

Following the release announcement of the 2.6.21 Linux kernel [story], Adrian Bunk noted that he no longer planned to track regressions [story]. He explained, "if we would take 'no regressions' seriously, it might take 4 or 5 months between releases due to the lack of developer manpower for handling regressions. But that should be considered OK if avoiding regressions was considered more important than getting as quick as possible to the next two week regression-merge window."

Linus Torvalds disagreed with Adrian's view that increasing the length of the release cycle would improve stability, "regressions _increase_ with longer release cycles. They don't get fewer." He went on to add, "you are ignoring the reality of development. The reality is that you have to balance things. If you have a four-month release cycle, where three and a half months are just 'wait for reports to trickle in from testers', you simply won't get _anything_ done. People will throw their hands up in frustration and go somewhere else." He continued:

"Do you really think bugs get fixed faster just because there wasn't a release? Quite the reverse. Bugs get _found_ faster thanks to a release (simply because you tend to get more information thanks to more users), giving the stable people more information, causing the bugs to be able to be found and fixed _more_quickly_ in the stable release than if we had waited for four months to release 2.6.21."

Announcing 2.6.21-rc2, Linus Torvalds noted, "I'm not very proud of this, because quite frankly, -rc2 has way more changes than I really like." The current Linux kernel development model is that the bulk of changes in a new kernel should happen during the -rc1 phase, with the rest of the -rc kernels being primarily bug fixes. Linus explains, "it's largely my fault, because I simply missed a V4L/DVB merge that came in before the merge window closed, but since I didn't notice it didn't make -rc1, and as such it got merged late and is in -rc2 instead." With typical humor he added, "but because I'll flail around wildly and rather blame anything else than my own incompetence, I'll just claim that all the other kernel developers have been irresponsible, and caused -rc2 to be bigger than needed. In some areas (you know who you are) it may even be true.."

Summarizing other changes in the new release candidate, Linus said, "apart from the V4L/DVB merge, we've got a late PARISC update, and a number of driver updates (ata, networking, usb) changes. Along with the normal smattering of random stuff (core networking, selinux, infiniband, agp, mips, arm)." He then pointed to Adrian's regression lists [story] noting that some are fixed but there are more to go. The latest kernel can be downloaded from your nearest kernel.org mirror. You can browse through all the changes using the gitweb interface. Kernel Newbiews maintains a useful summary of all the changes going into the latest version of the Linux kernel.

Following the release of the 2.6.20 kernel [story] Andrew Morton [interview] posted a list of patches in his -mm kernel, summarizing for each his plans as to whether or not they will be pushed upstream for inclusion in the upcoming 2.6.21 kernel. Andrew commented, "I'm getting fed up of holding onto hundreds of patches against subsystem trees, sending them over and over again and seeing nothing happen. I sent 242 patches out to subsystem maintainers on Monday and look at what's still here." In response to some confusion as to what happens to these patches, he went on explain, "once a subsystem has a subsystem tree (git or quilt) I basically never merge anything which belongs to that tree. It's always originator->mm->subsystemtree->Linus".

Jens Axboe has been involved with Linux since 1993. 30 years old, he lives in Copenhagen, Denmark, and works as a Linux Kernel developer for Oracle. His block layer rewrite launched the 2.5 kernel development branch, a layer he continues to maintain and improve. Interested in most anything dealing with IO, he has introduced several new IO schedulers to the kernel, including the default CFQ, or Complete Fair Queuing scheduler.

In this interview, Jens talks about how he got interested in Linux, how he became the maintainer of the block layer and other block devices, and what's involved in being a maintainer. He describes his work on IO schedulers, offering an indepth look at the design and current status of the CFQ scheduler, including a peek at what's in store for the future. He conveys his excitement about the new splice IO model, explaining how it came about and how it works. And he discusses the current 2.6 kernel development process, the impact of git, and why the GPL is important to him.

Robert Day proposed a couple of new kernel code maturity configuration options for tagging code as either "deprecated" or "obsolete". He referenced earlier confusion around the attempt to remove devfs [story] in which it wasn't clear on the current state and future plans for the code. He explained, "using deprecated code is still technically fine, but using obsolete code should be something that raises a red flag of some kind." Aside from a little confusion between the differences in definition between these two words, general feedback was positive. H. Peter Anvin supported the patch, "if nothing else, it gives some middle-of-the-roadness to the continual 'to remove or not to remove' debate." Robert also noted that the "deprecated" flag would be a useful sanity check when building a kernel, "this would seem to be a quick and dirty way to prune anything that is *supposed* to be obsolete from the build, to make sure you're not picking up dead code by accident." The patch includes definitions of these two states:

"Code that is tagged as 'deprecated' is officially still available for use but will typically have already been scheduled for removal at some point, so it's in your best interests to start looking for an alternative.

"Code that is tagged as 'obsolete' is officially no longer supported and shouldn't play a part in any normal build, but those features might still be available if you absolutely need access to them. You are *strongly* discouraged from continuing to depend on obsolete code on an ongoing, long-term basis."

Andrew Morton [interview] posted his patch queue with numerous comments about merge plans into the mainline kernel. Among his comments he noted that he would not yet be merging the Reiser4 filesystem [story], "reiser4. I was planning on merging this, but the batch_write/writev problemight wreck things, and I don't think the patches arising from my recent partial review have come through yet. So it's looking more like 2.6.20."

A large discussion followed Andrew's posting that focused on the current kernel development process [story]. Andrew expressed his concerns on what's currently happening, "people seem to treat the stabilisation period as a wonderful quiet time in which to run off and develop new features, rather than participating in the stabilisation. This has the following effects: 1: release cycles get longer 2: the kernel has more bugs 3: we put new features into the kernel faster than we otherwise would (see 2:, above)." Alan Cox [interview] proposed an idea, "a suggestion from the department of evil ideas: Call even cycles development odd ones stabilizing. Nothing gets into an odd one without a review and linux-kernel signoff/ack?" Linus Torvalds replied favorably, going on to note that he was surprised at how well the decision to only accept big merges in the two weeks following a major release has been accepted, "I actually expected people to dislike arbitrary rules more than they do, but I've come to believe that people _like_ having rules that they have to obey, as long as it's not a big pain for them. In other words, arbitrary rules are not actually disliked at all, people actually _like_ them, because suddenly there's less need for making unnecessary judgement decisions." Linus went on to spell out the idea further, "2.6.<odd> is 'the big initial merges with all the obvious fixes to make it all work' (ie roughly the current -rc2 or perhaps -rc3). 2.6.<even> is 'no big merges, just careful fixes' (ie the current 'real release')". He went on to caution:

"That said, I think Andrew was of the opinion that it doesn't really _fix_ anything, and he may well be right. What's the point of the odd release, if the weekly snapshots after that are supposed to be strictly better than it anyway? So I think I may like it just because it _seems_ to combine the good features of both the old naming scheme and the current one, but I suspect Andrew may be right in that it doesn't _really_ change anything, deep down."

With the release of the 2.6.16 Linux kernel, Adrian Bunk reiterated his previously debated intention of maintaining the 2.6.16.y kernel tree well into the future. The first 2.6.x.y release was 2.6.8.1 by Linus Torvalds [story], a quick one line fix for NFS. The idea was revisted a few months later in October of 2004 [story], but didn't actually gain momentum until March of 2005 [story] [story]. Beginning with the 2.6.11 kernel, the process was formalized with Greg KH and Chris Wright officially maintaining 2.6.x.y releases [story] until 2.6.(x+2) is released. For example, stable patches will be applied to the current 2.6.16.y kernel by Greg and Chris until 2.6.18 is released sometime well in the future.

Adrian's plan is to pick up the development of the 2.6.16.y kernel at that point, maintaining it much as the 2.4 kernel tree is is maintained [interview]. His intention is to maintain the tree as long is it is used and people contribute patches. The earlier debate on this idea was met with mixed reactions. At that time Greg KH cautioned, "the time and energy to do this for a long period of time is huge. If I were you, I would listen to the people who have and do maintain these kinds of kernels, it's not a simple job by any means."

In an email titled "read my lips: no more merges", Linux creator Linus Torvalds announced that the feature freeze, part of the newly improved development process [story], is now in effect for the 2.6.14 kernel. "Ok, it's been two weeks (actually, two weeks and one day) since 2.6.13, and that means that the merge window is closed," Linus explained. "I've released a 2.6.14-rc1, and we're now all supposed to help just clean up and fix everything, and aim for a really solid 2.6.14 release." He went on to add, "be nice now, and follow the rules: put away the new toys, and instead work on making sure the stuff that got merged is all solid. Ok?"

As for what was merged, Linus noted that there was "a lot of stuff all over the place." He began by pointing out that "pretty much every architecture got some updates," including alpha, arm, x86, x86-64, ppc, ia64, mips, and sparc. There was also "an absolutely _huge_ ACPI diff, largely because of some re-indentation." Other subsystems listed as receiving updates include drm, watchdog, hwmon, i2c, infiniband, input layer, md, dvb, v4l, pci, pcmcia, scsi, usb, sound driver, and network, "people may appreciate that the most common wireless network drivers got merged - centrino support is now in the standard kernel." Finally, Linus also noted, "on the filesystem level, FUSE got merged, and ntfs and xfs got updated. In the core VFS layer, the 'struct files' thing is now handled with RCU and has less expensive locking."

Linux creator Linus Torvalds sent a reminder to the Linux Kernel Mailing List that the merge window for 2.6.14 is coming to and end. "As per the new merge policies that were discussed during LKS in Ottawa earlier during the summer," Linus explained, "I'm going to accept new stuff for 2.6.14 only during the first two weeks after 2.6.13 was released." The new development policy was first discussed on the lkml with the release of 2.6.13-rc4 [story], and further elaborated with the release of 2.6.13 [story].

The 2.6.13 stable kernel was released on August 28'th [story]. "That release was ten days ago," Linus said, "so you've got four more days before I don't want any big merges." He went on to note that following the merge cutoff 2.6.14-rc1 will be released. "We certainly already have enough for 2.6.14," Linus noted, "but I just wanted to remind people that if they expected me to merge your work, you're getting closer to the cut-off point."

Back from the 2005 Linux Kernel Developers' Summit, Linus Torvalds released the 2.6.13-rc4 kernel. Linus noted that the improved development process discussed at the recent summit will begin after the upcoming release of the 2.6.13 kernel, "which is hopefully not too far away." The general idea of the new process, which improves upon last year's development model [story], is to require that all major merges happen within two weeks of a stable kernel release. All the rest of the time between releases should then be spent on fixing bugs. Linus summarized:

"So if you have a favourite kernel developer, please wake him up with a friendly kick to the head and explain this concept to him in small easy-to-understand words, and tell him that we're in the freeze process for 2.6.13 now, and that he should be gathering up the patches, and make sure they get to me _after_ 2.6.13 is out, but at that point do it in a timely manner."

Following SCO's allegations regarding the origination of some source code files comprising the Linux Kernel, in May of 2004 Linux creator Linus Torvalds implemented a simple method for tracking how patches reach the source tree [story]. The simple system was further refined in the following months [story], and has become second nature to most kernel developers. However, a recent debate on the lkml illustrated the fact that nothing is simple, in this case with concerns that archiving someone else's email address in the "Signed-off-by:" line could violate the UK's Data Protection Act.

Alan Cox [interview] suggested that to solve for this concern, the DCO, or Developer's Certificate of Origin, be updated to explicitly give permission to include an email address when archiving patch information. Linus agreed, "yes, I'll update the SubmittingPatches [documentation file] to state explicitly that the sign-off is a public record." Alan pointed out that adding a comment to the file alone is not enough, but that the new wording needs to be part of the DCO, "you have to -actively- agree to the DCO to submit a change, and that is what makes it work (whether you put something in submitting patches or not that is more explanatory)." Again, Linus agreed, "I'll also run it past the OSDL lawyer, and if others were to run it past their lawyers, that would be good." Once approved, the update will become version 1.1 of the DCO.