Mike Fedyk asked on the lkml why the -aa VM update hasn't gone into 2.5 yet. Andrew Morton, who recently split the large -aa update into smaller pieces for 2.4 inclusion explained that now was the wrong time for such an effort. Andrew's current focus in 2.5 is the buffer layer, and because of the intamacy between this layer and the VM, adding the -aa patches "would represent some wasted effort."
Also mentioned was the possibilty that 2.5 will use Rik van Riel's rmap VM, and hence tuning the existing VM would again be "wasted effort". In summary, Andrew offered, "So. My vote would be that unless the VM is actually impeding developers who are working on other parts of the kernel (it is not) then just leave it as-is for the while.".
It was also pointed out that the -aa patches are currently being merged into the 2.4 stable kernel.
From: Mike Fedyk To: linux-kernel Subject: -aa VM updates for 2.5 Date: Sat, 13 Apr 2002 16:39:06 -0700 Why haven't any of the -aa VM updates gone into 2.5? Especially after Andrew Morton has split it up this is surprising... From: Andrew Morton Subject: Re: -aa VM updates for 2.5 Date: Sat, 13 Apr 2002 16:55:11 -0700 I don't think there's really any point in doing that. None of the regular VM guys are really working 2.5 at this time. VM has a close relationship with buffers, so tinkering with the VM while I'm busily driving a truck through the buffer layer and setting up new writeback mechanisms would represent some wasted effort. We don't know yet whether 2.5 will have a reverse-mapping VM. If it does, then maintenance work against the current one is wasted effort and more patching pain. (I'd also like to investigate the option of not throttling page allocators by making them wait on I/O - make them wait on pages coming free instead). So. My vote would be that unless the VM is actually impeding developers who are working on other parts of the kernel (it is not) then just leave it as-is for the while. From: Mike Fedyk Subject: Re: -aa VM updates for 2.5 Date: Sat, 13 Apr 2002 17:13:23 -0700 Note: I'm just taking about the VM-xx patch from -aa, not some of the more controversial patches in -aa. On Sat, Apr 13, 2002 at 04:55:11PM -0700, Andrew Morton wrote: > I don't think there's really any point in doing that. > > None of the regular VM guys are really working 2.5 at this time. > > VM has a close relationship with buffers, so tinkering > with the VM while I'm busily driving a truck through the > buffer layer and setting up new writeback mechanisms > would represent some wasted effort. Yep, make sense. Though, keeping in mind the changes that are in -aa may help when making changes to the buffer layer, and possibly less effort if a problem is already fixed in -aa but not in 2.5. > We don't know yet whether 2.5 will have a reverse-mapping > VM. If it does, then maintenance work against the current > one is wasted effort and more patching pain. It looks like most of the vm changes can just be dropped into 2.5 over a few -pres. Especially since few (none?) have argued that the -aa vm-patch causes regressions. > (I'd also like to investigate the option of not throttling > page allocators by making them wait on I/O - make them > wait on pages coming free instead). Sounds interesting. > So. My vote would be that unless the VM is actually impeding > developers who are working on other parts of the kernel (it > is not) then just leave it as-is for the while. What about the recent threads on swapping in 2.5? Merging the -aa vm-patch into 2.5 will allow people to develop on the best known -aa VM to date, and can reduce duplicated effort. Though, admittedly it doesn't make much sense to do the same work on 2.4 and 2.5 at the same time (vm patch merging). Maybe it'll just be forward ported from 2.4... Mike From: Andrea Arcangeli Subject: Re: -aa VM updates for 2.5 Date: Mon, 15 Apr 2002 01:15:01 +0200 personally I think it should definitely go into 2.5 too. there's nothing that prevents the rmap design to be put on top of my vm updates, the rmap _patch_ (not the _design_ logic) does lots more stuff than just implementing the rmap design, and the reason I disagree with it (the patch, not the rmap design) is that it also backouts important fixes as just pointed out in a past email. As for Andrew's pagecache changes if it won't be Andrew spending time adapting his changes to my vm, it will be me later porting my vm patch on top of his changes. Since my code runs right now in production on some of the most important highend boxes out there, I think it should have precendence, but hey, I don't really care if I've to be the one to do the porting work as far as they gets merged eventually. Of course if somebody cameup with a patch fixing better all the issues that my current vm patch is addressing, and plus if somebody can design a better vm algorithm that will prove faster under my current most important VM benchmark with 1.2G of SGA in swap during simulated real life DB workload, that will be a very great news and in such case I will be _very_ _very_ glad to cp vm-33.gz /dev/null and to replace the whole thing with his code. The fact is that in all the feedback I got so far I didn't seen anything that surpasses my vm-33 updates, certainly not mainline without them, certainly not the rmap patch either, and this is why I'm assuming vm-33 is the right thing to merge at this point in time into both 2.4 and 2.5. Doing that will first of all place a solid base to allow Rik to extract a strict rmap patch to benchmark strictly the rmap design without the other benchmark-wise-pollution of current rmap patch. Same goes for Andrew, if vm-33 would be just in mainline he would just hack on top of it. The longer it gets delayed the more wasted resources IMHO. Also note that from my part I'm finished with the vm in 2.4. Unless I get a bugreport I will not touch it further (except for cleanups that doesn't affect functionality, for example I forgot to delete the show_stack export after the dump_stack is been introduced by Andrew). the last series of benchmark I run didn't show regression or instability in the numbers, and they were fast, it's behaving as expected under all scenarios, no too many magics, all magics sysctl configurable at least. There's still the issue of the oom killer. Andrew's right about the ways to fix the possible oom deadlock but they will become quite ugly code, similar to the feature in 2.2 that sends sigterm to X first (that I'd like to forward port to 2. too). But I'm not very happy with the algorithm either. my highmem machine runs with 800M of swap free and the SGA used by the DB is 1.7G mapped by a dozen of tasks. Now if after some day of web browsing I hit a bug in knoqueror from kde head cvs the oom killer will start killing all the idle DB tasks attached to the SGA instead of killing konqueror. That's the wrong thing to do in such case, the probabilistic effort would do much better in such case, and it will get the other cases right too most of the time. BTW, (talking about being perfect in such area) the non-overcommit/beancounter work as well can be developed on top of vm-33 of course, only _then_ it will be safe to loop forever into the memory balancing without breaking the loop until we succeed in freeing ram (and even in such case the oom killer will be superflous, because all -ENOMEM mem failures will happen at the memstats/vma level). Andrea
From: bert hubert Subject: -aa VM updates for 2.4 Date: Sun, 14 Apr 2002 12:40:46 +0200 On Sat, Apr 13, 2002 at 11:56:26PM +0000, Andrew Morton wrote: > Mike Fedyk wrote: > > Why haven't any of the -aa VM updates gone into 2.5? Especially after Andrew > > Morton has split it up this is surprising... > > I don't think there's really any point in doing that. Are they going to be in 2.4 anytime soon? And if not, why not? Thanks. Regards, bert From: Mike Fedyk Subject: Re: -aa VM updates for 2.4 Date: Sun, 14 Apr 2002 06:22:39 -0700 Yes, some have alredy gone into 2.4.19-pre5 or pre6. The large vm patch in -aa was split up by Andrew Morton (as mentioned earlier in this thread). Expect the next batch to go in the next 19-pre or more likely 20-pre because many more people test the released versions than the pre kernels. Mike