login
Search
Linus Torvalds released the final 2.6.1 Linux kernel. This release has a lot of changes since 2.6.0,
many things have been merged from the -mm tree [forum]
This release includes a patch for the mremap() vulnerability too.
After releasing this Linus will be in Australia for the week, Linus explains:
"I'm going to
be in Australia (and on airplanes) for the week, but we're
all in the capable hands of Andrew, so why worry? The fact that I'm
fleeing the country should in no way be construed as anything sinister at
all, no siree. Nope. I'm innocent, and nobody saw me do it."
Date: Thu, 8 Jan 2004 22:49:01 -0800 (PST)
From: Linus Torvalds [email blocked]
To: Kernel Mailing List [email blocked]
Subject: Linux-2.6.1
Ok, the diffs from -rc3 are minimal, most noticeably the (very _very_ hard
to trigger, but nasty if you ever did) fork() race that Ingo found.
I'm going to be in Australia (and on airplanes) for the week, but we're
all in the capable hands of Andrew, so why worry? The fact that I'm
fleeing the country should in no way be construed as anything sinister at
all, no siree. Nope. I'm innocent, and nobody saw me do it.
The full changelog is getting uploaded right now along with the release,
and the BK trees have already been pushed.
Up up and away,
Linus
============================================
Summary of changes from v2.6.1-rc3 to v2.6.1
============================================
Dave Jones:
o [AGPGART] Duh, is_r200 is a function, not a variable
Linus Torvalds:
o Fix subtle fork() race that Ingo noticed
Nathan Scott:
o [XFS] Add the noikeep mount option, make ikeep the default for now
o [XFS] Fix a possible bio-leak on I/O submission, in a case where no
I/O was required
o [XFS] Update XFS documentation
-mm1
2.6.1-mm1 has been released too.
no rc4?
I think it is lame to release 2.6.1 without testing these fixes in a 2.6.1-rc4 release. Ideally, a 2.6.x release should be IDENTICAL to the most recent 2.6.x-rc.
No, not this time...
If you look at the changelog, it looks that really only very simple stuff was done from rc3 -> release.
It was necessary to really push 2.6.1 because of the mremap() issue. Creating a rc4 that only has these minor fixes (which are not likely to break anything) would have delayed the release, and not many people would have tested it anyway... because: I would not compile a new kernel if nothing important has changed anyhow (with respect to what I already have running).
Re: No, not this time...
While I agree with you that 2.6.1 needed to get out the door due to the mremap() fix, I don't agree with you that not very many people would have tested it as -rc4. Your argument goes like "I wouldn't have tested it, so not many others would have either" - that's a flawed argument.
I for one rutinely build and test the all the new kernel releases no matter if they contain fixes relevant to me or not, and I'd suspect many other people do so as well - just look at reports to lkml every time a new -rc comes out.
Hum...
This may be true for many geeks home machines. However, it's unlikely that server administrators upgrade to each -rc. Even if it doesn't contain "fixes relevant to them", they would take the risk to discover something is broken because of a minor fix.
many geeks....
do have enough machines at home to test that stuff well.
you should'nt even have to upgrade your production machines, it's good to have ~identical machines to test new stuff on, before upgrading production machines ;)
Flawed??
That is certainly not a flawed argument. If you want a more formal explination, assume that N represents the number of potential testers, and that this set does not change for -pre, -rc, and -final candidates. This implies that each potential tester is a Linux user and the the number of Linux users between releases is static. I'll point out that this is not a valid general assumption, but is representitive small timeframes, such as "needed to get out the door" ...
Now, if the person who would not have tested it is a member of the N set and admits such, then the number of available testers must be smaller because N - 1 < N, thus "not as many". The poster, therefore, is fundamentally correct, within the constraints of the given assumptions (specifically, that N(t0) = N(t1)).
Now, we can assume that N(t1) > N(t0) because Linux is becoming more popular, no? So there are more potential testers. So the question is whether the percentage of actual testers (specifically of the -pre or -rc variety, and not just -final) increases faster than the loss of non-final testers (such as the one above) in the [t0..t1] interval. Unfortunately we cannot derive real numbers for this, so we have to use estimation and speculation.
I, personally, have been a long-time Linux user (~1995-1997) and only regularly have compiled -final releases. In general, I only patch the kernel when I want/need certain features. My observations show that most Linux users use either -final or simply whatever the vendors of their distro ships. But this is not scientific.
So let's break this argument down one more level. Poster 1 says "I only test -final, therefore there are fewer testers for -pre and -rc kernels." Your response is that this is a generalization and is therefore invalid. To counter, you state "I compile every release, -pre, -rc, or -final, and therefore so does everyone else."
The fallacy here should be obvious...
2.6 vs 2.4+preempt/lowlatency
Not sure if this is the best place to ask, but...
How good is 2.6 for the desktop compared to 2.4 + preempt + lowlatency + other Con Kolivas patches?
It's the same....
It's the same....
I've noticed otherwise
When jumping from 2.4 to 2.6, even with pre-emptive enabled in both, I've noticed better response in 3D games and running multiple processes at once. The cpu gets notched up higher in 2.6, but the result is a really smooth desktop.
I can't really say why, all I know is that the thing doesn't wanna drop processes.. like say running xmms, compiling a kernel, downloading 2.5 million newsgroup headers, opening mozilla, reading email- etc. It does choke at an inteval of about 30 seconds, but it doesn't let your desktop crawl other than that. 2.4 would just freeze, and you'd get to wait for it to let you affect X again.
Anyhow, that's what I've noticed. The how's and why's I don't understand, and until I get paid to program rather than sysadmin, I don't care. Just my take Jake. Later.
2.4.23 >
Well, the nice thing about anything after 2.4.23 is the 0(1) Scheduler.
2.6 already has this and it was its greatest move, IMHO. But I'm runnign 2.4.25-pre5 and its running great for me. I played with 2.6 for a while, but decided I wasn't quite ready for it. And honestly, I never saw much of a difference in everyday usage. Not knocking 2.6 here, I think its great, but I am personally waiting for 2.6.4 ish Little bugs and traps that had been overlooked will be fixed and it will become "stable" at this point.
Ok ok, I know its considered "stable" at this point, but that is relative. I mean, I wouldn't put 2.6 on a production server as of yet. Just my personal opinion (not trying to hate).
O(1)
2.4.x DO NOT have O(1) scheduler, and won't have it ever.
It's 2.6 thing.
Also, its O(1) (capital "oh"), not 0(1) (zero).
--
:wq
Well poo on me. I must have
Well poo on me. I must have been smoking crack but I swore to myself in 2.4.23 that they back ported O(1) to 2.4. I looked at the change log and couldn't find it.
Now I'm forced to see whats holding up openMosix again since 2.4.23. I apologize for my ignorance.
The scheduler is nice and wel
The scheduler is nice and well, but for me the nicest features of 2.6 are epoll and futexes.
Better
Better all round for desktop through to server. Only scenario where 2.4 is better is desktop machines with less than 128Mb Ram.
but
But isn't 2.6 supposed to have a vastly improved VM system? I would guess that 2.6 is much better than 2.4 with low ram.
No
Afraid not. Test cases show exactly the opposite. That is currently under investigation but unfortunately not a high priority since hardly any newer machines have less than 128Mb ram. Note I did not say this was satisfactory.
Swapping?
Isn't 2.6's VM's better, and didn't 2.6 also started swapping later, but when it does all performance is gone? Or is bad swap performance another, seperate problem?
Swapping part issue
While swap is hit less frequently in 2.6 you can get into strange swapstorms where nothing ever seems to spend enough time in physical ram to do any real work once you do start. This shows up on 32Mb boxes. Making it as unswappy as possible doesn't help either as the vm gets into a continuous cycle of try to allocate ram, fail and retry without ever going anywhere. No balance point of "swappiness" can be obtained by tuning this alone; something about the vm makes it suffer on low memory machines and some people are investigating it. While the 2.6 vm is better most of the time, there are corner cases on low memory machines where the 2.4 vm beats it.
fairness
Currently, everything is pointing to fairness as the major problem. Everything struggles along at an equal rate in 2.6 while in 2.4 one process will be allowed to hog lots of memory and speedily finish its job, freeing memory for others, etc.
Its a fairly sticky problem.
Then it seems to be not reall
Then it seems to be not really a VM problem, but more a scheduler problem. If that's really the case then it seems to be relatively easy to solve. For instance by switching to another scheduler policy when all ram is used (e.g. when a process asks for memory that isn't there, then put it to sleep until some other process frees some memory, with a timeout to avoid disasters ;). Maybe a special low ram scheduler is a good idea, but probably rather hard to make. The most tricky part seems to be how to detect which process uses a lot of ram only for short moments, and give those the most priority when they run. Maybe a seperate scheduler for ram usage? Which moves processes' memory to and from swap, and works together with the normal scheduler.
VM Scheduler
You can't blame the cpu scheduler for this. The vm works virtually independent of the cpu scheduler. However vm scheduling is a very real option but a big task. There is no point tackling this with a vm scheduler unless it's proven that this is exactly what is at fault or else it may not do anything useful but impose further overhead. Finding exactly what the cause is would be the only useful thing and someone's working on it. Hopefully they'll find something useful before they give up and we lose the ability to run 2.6 on low memory desktops.
What I meant was, that if the
What I meant was, that if the problem is more or less caused by "fairness" as the comment I replied to says, then it's probably indirectly caused by the scheduler because it's main purpose is "fairness". The programs that get cpu are also the programs that use memory. Simply put: programs that do not run can't ask for more memory.
If a system is really low on memory, that is, if there is more active memory than there is real ram, then it's much better to schedule by memory usage/placement than by cpu usage. In other words, scheduling by cpu is normally nice and good, but if using swap extensively it doesn't make much sense anymore, because the major bottleneck is hd speed. Then a good scheduler is maybe only counterproductive. What I basically said was that the scheduler shouldn't be totally VM unaware. For instance by somehow telling the scheduler to give very low priority to programs who's active memory is swapped out, while also switching the swapped out programs regularly.
If the problem is really caused by the VM then the VM moves memory too soon to swap, or does in general a bad job at guessing which memory can be moved to swap. Or something else more subtle that causes problems. But the theory that because more processes get cpu time, more ram is actively used in a shorter timeframe, thus causing bad swap behaviour seems plausible. What counts is how much active memory there is, the inactive memory can be easily swapped out. Thus if the 2.6 kernel somehow has more active memory then it will perform badly when ram is scarce. It can be easily verified if this is the case or not.
A low memory situation where swap must be used to swap out active memory should or never happen (out of memory errors, oomkiller, etc), or must be handled specially because it's so totally different. I'm no kernel hacker, I don't know the internals of the VM or the cpu scheduler, I'm just using common sense and guessing around.
VM
I understood your first comment. The newer cpu scheduler is not fairer than the old one (so feel free to postulate that it's less fair and that is responsible but I doubt it).
There is only so much information being piped around the kernel from one subsystem to another and the cpu scheduler knows basically nothing about the vm. While in an ideal world it would know what every subsystem was doing, the overhead would be massive and requires a complete rewrite.
If you've seen my comments before you'd know I'm a big proponent of feedback and autoregulation. In this case, however, the question is: what exactly should the cpu scheduler do with the information from the vm? The vm doesn't even know what to do with it's own feedback information and there is enough argument over that. I have a patch, for example, that tries to do some small feedback in the vm with swappiness but that isn't being considered for mainline.
Well, as I said, I think an e
Well, as I said, I think an extensive swapping situation shouldn't happen, except if it's handled specially. Does the 2.6 kernel also perform worse than 2.4 on low memory systems which have no swap at all? If not, then that would be encouraging.
I imagine that if active memory is swapped out, then there can happen a sort of chain reaction: A process asks for memory in swap, which needs to be copied to ram. In the meantime that process sleeps, and the scheduler runs other processes, who also have once in a while memory swapped out, thus causing more hd activity. Because there is simply not enough ram for all processes, the end result can be that all processes are causing extra swapping and working against eachother (e.g. one process starts retrieving memory from swap, then another forces it back to swap before it was actually used). I do not know how 2.4 or 2.6 prevent this kind of situations, but maybe 2.4 does it somehow better than 2.6.
The idea I had was that the vm put a process to sleep when it's active memory is swapped out, thus avoiding that it will be run. The scheduler doesn't have to know anything about the vm, as long as the vm can somehow prevent the scheduler to run certain tasks. I assume that the vm selects which memory to move to swap by activity, if so then it seems easy to add an extra check to see if the memory that needs to be moved now has more than the prefered activity. If that is the case then the point of extensive swap usage is more or less reached. Then the vm could enable a tiny simple memory scheduler, which puts processes with the lowest priority to sleep and swaps out all their memory, thus giving the other processes more space to breath. The goal is to prevent that any active process needs to get memory from swap (other than inactive memory that is suddenly used). All the swapping will be done in the background. The latency of swapped out processes will be very high, depending on how fast the hd is, but the overall performance should be much better.
Swapping
Fortunately it performs better on no-swap machines than 2.4. It is also less likely to hit swap if it is there, and performs better while hitting swap on decent memory machines. It's only the combination of a low memory machine where you're running tasks that take up most of the physical memory space and then hitting swap when the problem occurs. The test case is well established; the best way to tackle it is not.
And as I said, I understood your idea with the cpu scheduler, I just said it was the wrong way to tackle the problem. No point trying to stop a horse and cart by using breaks on the cart when the horse can slow down (if you get my drift).
It's more a too small horse w
It's more a too small horse with an overloaded cart. Drop some of the cargo and pick it up later so the cart can at least move. But that's not smart if the cart can still move, although somewhat slower, and the road is long.
Good to hear that without swap 2.6 is better than 2.4, I never liked swap anyway...
I think it's time to ignore me, I'm starting to get weird ideas. What about compressing data on the fly? It's hopefully faster than swapping, or could make swapping faster by compressing the data before swapping it out. Of course such data won't compress very well, to spoil everything.
I don't know about other programmers, but I get a bit tired of all the extra code to handle NULL from malloc gracefully, so just returning NULL when memory is tight would make all those extra code not for nothing. Currently it feels stupid to make those checks while knowing that some program will be killed when memory is really tight. It's simply saying "stop" earlier when loading the cart, solving the problem as soon as possible, giving an advantage to programs that handle out of memory situations well. Now one bad programmed bloaty program can spoil it for another well written program that knows how to handle NULL, which gets killed because it happens to use malloc or its not yet used virtual memory at the wrong time. Let the programs weed themselves out, big chance that the relative big ones die.
Compression?
Wow this thread is real hard to get the final word on! There is compressed caching for the vm but only for 2.4 (linuxcompressed.sf.net) and it hasn't been updated in a while and had lots of smp races and ram is dirt cheap and ...
Cheers
and ...
... the benchmark results aren't that great, and tested with a one gig cpu, while all low memory systems will be much slower.
If you want the final word then finish your sentence. ;-)
I'll shut up now. Thanks for clearing some things up,
good luck
Final word
.
2.6 with nForce2
Anybody know if they finally fixed the issues with 2.6 randomly crashing with APIC enabled on nForce2 motherboards? At first I thought this was fixed somewhere in the later 2.6.0-test kernels, but my 2.6.0 final kernel it seems I still have to turn it off, or else I get complete system lockups (even the SysRq keys) after a few hours of uptime.
dunno, works for me on 2 nfoc
dunno, works for me on 2 nfoce2 mobos (epox 8rda+ and 8rda3+), totally different systems, had problems with -test, but is fine since 2.6.0, no crashes, even after hours of load, and also apic is enabled (at least /proc/interrupts and kerneloutput makes me believe that)
hope that helped
There has been a lot of work
There has been a lot of work and testing done on this issue and it seems that we never really got the correct fix. -mm kernels had one set of fixes in for a little while, then were dropped out. Ross Dickinson also has done a lot of work on the issue too. Doing a search on lkml archive will get you tons of posts on the subject. Hard to say exactly what the correct fix is though, try them out and we would love to here your experiences.