It was recently pointed out that most of the x86 architecture patches had been merged into the mainline 2.6.25 kernel, except for the kgdb patches. Linus Torvalds replied, "I won't even consider pulling it unless it's offered as a separate tree, not mixed up with other things. At that point I can give a look." He continued:

"That said, I explained to Ingo why I'm not particularly interested in it. I don't think that 'developer-centric' debugging is really even remotely our problem, and that I'm personally a lot more interested in infrastructure that helps normal users give better bug-reports. And kgdb isn't even _remotely_ it.

"So I'd merge a patch that puts oops information (or the whole console printout) in the Intel management stuff in a heartbeat. That code is likely much grottier than any kgdb thing will ever be (Intel really screwed up the interface and made it some insane XML thing), but it's also fundamentally more important - if it means that normal users can give oops reports after they happened in X (or, these days, probably more commonly during suspend/resume) and the machine just died."

"The following patches have been in the -mm tree for a while, and I plan to push them to Linus when the 2.6.25 merge window opens," began Theodore Ts'o, offering the patches for review before they are merged. He explained that the patches introduce some of the final changes to the ext4 on-disk format, "ext4, shouldn't be deployed to production systems yet, although we do salute those who are willing to be guinea pigs and play with this code!" He continued:

"With this patch series, it is expected that [the] ext4 format should be settling down. We still have delayed allocation and online defrag which aren't quite ready to merge, but those shouldn't affect the on-disk format. I don't expect any other on-disk format changes to show up after this point, but I've been wrong before.... any such changes would have to have a Really Good Reason, though."

The final 2.6.24 Linux kernel is expected any day now, so the various subsystem maintainers have begun summarizing what changes are expected to be merged into the mainline kernel during the 2.6.25 merge window. Ingo Molnar spoke to changes for the x86 architecture, "there are 763 commits in x86.git so far, from more than 90 contributors, so it would be difficult to mention and credit every contribution in this mail." Along with a lengthy list of other changes, he included:

"Continued, intense arch/x86 unification and cleanup work by lots of people; FIFO ticket spinlocks for better spinlock scalability; 'regset' generalizations - the most important step towards utrace support (==next-gen ptrace); support for more than 255 CPUs [up to 4096 - in theory up to 65535]; almost complete 64-bit paravirt guest support; KGDB support on x86, finally!"

"Yes, I know ... another tree, just what everyone wants," quipped James Bottomley, announcing his new merge candidate (-mc) tree:

"This one has a specific purpose: It's my tree tracking everyone else's git and quilt trees so I get early warning if there are going to be any merge issues. However, it struck me it might be useful to anyone wishing to track what's going upstream more closely."

James noted that his new tree is available in git, and being automatically built each night. "As you can see from the reverts and the skips, we have trouble even now (and that's after I fixed up most of the failures in SCSI). ACPI and the x86 trees clash hideously, so I kicked out x86. Jens' block tree has two patches which clash with Bart's ide quilt. Greg actually has one patch in his tree that clashes with one of mine." He also noted, "this tree is currently very storage centric (i.e. I haven't included net trees or quilts because I didn't think they'd be likely to clash with my SCSI trees). However, if it could be more generally useful, I could add other trees and quilts to it."

A recent attempt to push some V4L/DVB updates for inclusion in the 2.6.24 Linux kernel met with some resistance. Linus Torvalds summarized the problems affecting the em28xx video driver:

"I've talked to various people, and none of the main kernel people end up being at all interested in a kernel that has external dependencies on binary blobs for tuners.

"So right now it seems like while I would personally want to have more vendors supprt their own drivers, if that in this case means that we'd have to have user-space and unmaintainable binaries to tune the cards, everybody seems to hate that idea."

Ralf Baechle posted the Linux/MIPS architecture merge plans for the upcoming 2.6.24 kernel. The diffstat for all changes showed, "435 files changed, 14274 insertions(+), 10196 deletions(-)", about which Ralf noted, "the number of patch lines and files is inflated by two large whitespace cleanup patches." He continued:

"The biggest actual changes are the support for tickless kernels on MIPS and the rewrite for many of the timer devices previously used as clocksources and clockevents. Various cleanups, including some moving of code and support for 32-bit Broadcom BCM47XX processors, the return of support for LASAT which isn't quite as unused as previously thought."

Roland Dreier posted an updated overview of his InfiniBand driver merge plans during the upcoming 2.6.24-rc1 merge window. The InfiniBand driver was first merged into the 2.6.11 Linux kernel. Roland noted, "since 2.6.23 still isn't out, and I've managed to reduce my patch review backlog a bit, it's probably a good idea to give another update about what I have queued for 2.6.24 already and what I hope to get to before the merge window opens." His description of the upcoming merge queue is broken into three sections, one for each subdirectory within drivers/infiniband. In addition to listing all patches intended for 2.6.24, Roland also listed two issues that would most likely wait for 2.6.25:

"1) Multiple CQ event vector support. I haven't seen any discussions about how ULPs or userspace apps should decide which vector to use, and hence no progress has been made since we deferred this during the 2.6.23 merge window.

"2) XRC. Given the length of the backlog above and the fact that a first draft of this code has not been posted yet, I don't see any way that we could have something this major ready in time."

Trond Myklebust noted the NFS client updates for the upcoming 2.6.24 kernel:

"Aside from the usual updates from Chuck for NFS-over-IPv6 (still incomplete) and a number of bugfixes for the text-based mount code, the main news in the NFS tree is the merging of support for the NFS/RDMA client code from Tom Talpey and the NetApp New England (NANE) team."

He continued, "we also have the 64-bit inode support from RedHat/Peter Staubach. There is also the addition of a nfs_vm_page_mkwrite() method in order to clean up the mmap() write code. Finally, I've been working on a number of updates for the attribute revalidation, having pulled apart most of the dentry and attribute revalidation into separate variables. A number of fixes that address existing bugs fell out of that review, which should hopefully result in more efficient dcache behaviour..." Actual source changes can be browsed in the NFS client git repository.

"I've just released the 2.6.23-rc9-ext4-1. It collapses some patches in preparation for pushing them to Linus, and adds some of the cleanup patches that had been incorporated into Andrew's broken-out-2007-10-01-04-09 series," announced Theodore Ts'o. He also noted of the current ext4 git tree, "it also has some new development patches in the unstable (not yet ready to push to mainline) portion of the patch series." In an earlier thread Theodore posted a series of patches specifically intended for inclusion in the upcoming 2.6.24 kernel. Included in the patch series was a patch for improving fsck performance, "in performance tests testing e2fsck time, we have seen that e2fsck time on ext3 grows linearly with the total number of inodes in the filesytem. In ext4 with the uninitialized block groups feature, the e2fsck time is constant, based solely on the number of used inodes rather than the total inode count." The patch included an explanation of how the feature works, enabled through a mkfs option:

"With this feature, there is a a high water mark of used inodes for each block group. Block and inode bitmaps can be uninitialized on disk via a flag in the group descriptor to avoid reading or scanning them at e2fsck time. A checksum of each group descriptor is used to ensure that corruption in the group descriptor's bit flags does not cause incorrect operation."

"I'm a bit behind after investigating the TCP performance issues that turned out to be HW specific problems. It's a bit of a disappointment, I thought maybe there was a cool bug to fix in TCP :-)" explained David Miller, posting his networking merge plans for the upcoming 2.6.24 kernel. He noted, "I merged in Jeff Garzik's and John Linville's latest and I'm running the current tree on my workstation most of today with good results so far." David added, "I plan to commit my Neptune driver in it's current state, and that's the last new feature going in."

In an earlier discussion last month on the Linux netdev mailing list, David described how many changes were in his net-2.6.24 git tree, "it's to the point where every single bug fix put into Linus's tree creates a merge conflict with net-2.6.24, we are simply touching that much stuff. :-)" He added, "we've touched so much in net-2.6.24 that we really should be auditing the thing and fixing any bugs that have been added. If you're bored and looking for something to do, pick an odd NAPI driver and audit it in the net-2.6.24 tree."

"I said I was hoping that -rc8 was the last -rc, and I hate doing this, but we've had more changes since -rc8 than we had in -rc8. And while most of them are pretty trivial, I really couldn't face doing a 2.6.23 release and take the risk of some really stupid brown-paper-bag thing," Linus Torvalds said, announcing the 2.6.23-rc9 kernel. He added, "so there's a final -rc out there, and right now my plan is to make this series really short, and release 2.6.23 in a few days. So please do give it a last good testing, and holler about any issues you find!" He went on to warn developers that the first thing planned for 2.6.24 was to merge the unified x86 architecture:

"This is also a good time to warn about the fact that we're doing the x86 merge very soon (as in the next day or two) after 2.6.23 is out, so if you have pending patches for the next series that touch arch/i386 or x86-64, you should get in touch with Thomas Gleixner and Ingo Molnar, who are the keepers of the merge scripts, and will help you prepare..

"Doing it as early as possible in the 2.6.24-rc4 series (basically I'll do it first thing) will mean that we'll have the maximum amount of time to sort out any issues, and the thing is, Thomas and Ingo already have a tree ready to go, so people can check their work against that, and don't need to think that they have to do any fixups after it his *my* tree. It would be much better if everybody was just ready for it, and not taken by surprise."

With the official release of the 2.6.23 kernel expected any day now, Andrew Morton posted his -mm merge plans for the 2.6.24 kernel. The current Linux kernel development model is to open up the mainline kernel for significant merges during the two weeks following a major kernel release. Thus, during the two weeks following the imminent release of the 2.6.23 kernel, subsystem maintainers will push their latest trees to Linus' mainline tree. Andrew Morton will also push many of the patches he collects in his -mm tree to Linus' mainline tree during these two weeks, as detailed in his email. At the end of the merge window, 2.6.24-rc1 will be released and the stabilization process begins, though in reality significant merges also often slip in between -rc1 and -rc2. A series of -rc kernels will be released, eventually leading to a stable 2.6.24 kernel two or three months after the process started, and it all starts again.

"With 2.6.24 probably opening in the not-too-distant future, it's probably a good time to review what my plans are for when the merge window opens," began Roland Dreier on the Linux Kernel mailing list. He reflected on the recent decision to phase in usage of reviewed-by tags noting that he was a little behind on reviews, "unfortunately, due to the length of the backlog and the fact that 2.6.23 seems fairly close, some of the things listed below are going to miss the 2.6.24 merge window." Roland continued, stressing the importance of getting in-depth reviews:

"Although the plan is to phase in requiring 'Reviewed-by:' gently, for this merge, if you can get someone other than me to review your work, then the chances of it being merged increase dramatically. I'm talking about a real review -- ideally, someone independent (from another company would be good) who is willing to provide a 'Reviewed-by:' line that means the reviewer has really looked at and thought about the patch. There should be a mailing list thread you can point me at where the reviewer comments on the patch and a new version of that patch addressing all comments is posted (or in exceptional cases, where the patch is perfect to start with, where the reviewer says the patch is great)."

Included in Andrew Morton's potential 2.6.23 merge list [story] were a series of patches to make the x86-64 architecture tickless. Andi Kleen, the x86-64 maintainer replied, "I'm sceptical about the dynticks code. It just rips out the x86-64 timing code completely, which needs a lot more review and testing. Probably not .23." Linus Torvalds agreed, "we are *not* going to do another 'rip everything out, and replace it with new code' again. Over my dead body. We're going to do this thing gradually, or not at all." He went on to explain "the patch-set itself actually looks fine, as far as I'm concerned. But it does seem to have that 'enable everything in one go' problem. I'd much rather see one time source at a time being converted, and enabled then and there, so that when people report problems and do a bisection, if it was HPET that broke, you get the commit that changed HPET."

In response to the pains caused by the original dyntick merge in 2.6.21, Ingo Molnar acknowledged, "we had 12 -hrt/dynticks merge related regressions between 2.6.21-rc1 and -final, and 4 after final." He went on to point out, "it's all pretty quiet today on the dynticks regressions front. (there are no open regressions in either the upstream i386 code or in the devel patches we are aware of)." As to the source of the bugs, he explained, "the majority of the above bugs were in the infrastructure code. (the worst was the generic resume/suspend one fixed in 2.6.21.2) And sadly, a fair number of the infrastructure bugs we introduced during the frentic clockevents/dynticks rewrites/redesigns we did between .20 and .21. That was a royally stupid mistake for us to do - instead of patiently waiting for the bugs to be shaken out we destabilized the infrastructure. (it was a 'lets make this thing so nice that it's impossible to reject' instintic gut reaction.)" Linus replied, "one thing I'll happily talk about is that while 2.6.21 was painful, you and Thomas in particular were both very responsible about the thing, so no, I'm not at all complaining or worried about it in that sense! I just really _really_ wish we could have two fairly stable releases in a row. I think 2.6.22 has the potential to be a pretty good setup, and I'd really like to avoid having another 2.6.21 immediately afterwards."

Following the release of the 2.6.22 kernel [story], Andrew Morton [interview] posted a list of a wide range of patches that are in his -mm kernel, summarizing for each his plans as to whether or not they will be pushed upstream for inclusion in the upcoming 2.6.23 kernel. Comments included simply noting "merge" or "hold", as well as "these appear to need some work,", "don't know, need to ping suitable developers over this work," and "sent to maintainer." Perhaps most entertaining was Andrew's response to the vmscan-give-referenced-active-and-unmapped-pages-a-second-trip-around-the-lru.patch, "this is scary. Will sit and admire it until it has been demonstrated to be a net gain." It is possible to track which patches are actually merged using the gitweb interface to Linus' kernel tree.