Where can I get a 4G/4G patch for a recent kernel (>= 2.6.16) ? I've got some dual xeons with a ton of RAM that're running out of lowmem with the stock kernel (which I understand has a 1/3 split).
I was also looking for a newer version of 4G/4G patch, even asked Ingo, but all to no avail.
Try 2G/2G split (CONFIG_VMSPLIT_2G) or even 1G/3G (CONFIG_VMSPLIT_1G) if it works for your workload. Also, enable pagetables in highmem (CONFIG_HIGHPTE). That should help a little or might even resolve all your troubles.
Are these config options available in 2.6.17 kernel?
I downloaded 2.6.17 upgrade kernel and I could not locate the CONFIG option. Could you please point me how to use 2G/2G or 3G/1G options.
Thanks
Kumar
Also, It seems to work like charm for me. But I am bit nervous now.
Situation is like as follows -
My FC4 machine have 2 GB RAM. Eralier it had High Mem enabled and system can see full 2GB memory (1GB low mem and 1GB high mem).
I had a non-root user program that was opening 100's of thosands of sockets.
As the program runs, I can see free low mem keep on reducing and upltimately becomes 0 (while full 1 GB high mem of RAM is still available). Around this time OOM kicks in and start killing some process and system hangs. (A simple user process brough the system to halt!. I was rather expecting new sockets calls to fail and system integrity maintained. Also, I would have expected swapping to kick in before OOM).
After using 2G/2G split, 2 GB is lowmem and 0 highmem. In this case, my socket opening program could create as many sockets and could use whole of 2 GB RAM. Also, once mem reaches around 0, sockets opening starts failing and I also see swapping also kicks in. No OOM business or no system hanging!. All seems great like one one would expect from a solid kernel. I can run a user process malloc'ing several chunks. I could very well alloacte Gig's of mem.
As i understand (I am no kernel expert, just an application programmer making extensive use of system calls), low mem can be used by kernel or user space equally well while high mem can be used by user space programs only. Is my understanding correct? If that is the case, what is the point in having any mem as highmem (if that is possible to configure!).
Since, 2G/2G split seem to be hidden so deep down, i am sure there are some caveats and i am not aware of some dangerous implications.
As i understand (I am no kernel expert, just an application programmer making extensive use of system calls), low mem can be used by kernel or user space equally well while high mem can be used by user space programs only. Is my understanding correct?
Yes, lowmem (normal mem) can be used by kernel space or userspace. Highmem can be used in kernelspace, but requires lots of trickery (so not many kernel apps use it). That said.. the 1/3 2/2 3/1 split isn't a split of the real memory, its a split of the address space (that is the ram visible to a process). So for any given process, the addresses in the first 1/2/3 gigs are addresses and can only be used by the kernel, while the rest is userspace. So if you have a 3/1 split, the application cannot mmap()/malloc() more than 1 gig (or 3gigs if you have a 3/1 split).. Also, the kernelspace part of the memory is the same for all processes, while the userspace part is obviously not the same.
If that is the case, what is the point in having any mem as highmem (if that is possible to configure!).
Well, if you have 2 gigs of ram.. you can use a 2/2 split.. and you wont have any highmem, but then your application is limited to mapping 2 gigs of other stuff. So lets says its an opengl app and you want to map part of the video card memory into you address space (lets say 500 megs) then you can only malloc 1.5 gig because you run out of addresses. If you have a 1/3 split, you app could mmap() 3 gigs so 2 gig could be in physical memory and the rest is in the swap (if its been written to).
The easy solution to that problem is to use 64bit addresses. So if you use amd64 or any other 64bit architecture, you dont have that problem. Since you have a very large address space, you can map anything anywhere and you could even map everything everywhere if you wanted...
> Since, 2G/2G split seem to be hidden so deep down, i am sure there
> are some caveats and i am not aware of some dangerous implications.
AFAIK, the only downside is that now your userspace programs/processes have slightly less virtual memory available to them.
IIRC, with standard 3/1 split you have something like 2.5GB virtual space available for every process - that is how big virtual memory size any of your processes can reach (by using mmap() or allocating memory from the heap).
When you configure kernel with 2/2 split, available virtual memory per process drops to something like 1.5GB (1GB less).
So it's up to you, if you have any really big/fat processes (typically some databases, Oracle comes to mind) 2/2 won't be a good choice. But in most cases/workloads you wouldn't have such heavy processes, so IMHO in those cases using 2/2 split doesn't have any downside, but only positive effects - simpler, faster and more fair memory management.
> AFAIK, the only downside is that now your userspace programs/processes have slightly less virtual memory available to them.
Oh well, here's the first application I have just encountered that has problems with non-default kernel/user split:
Wine called with no arguments.
Invoking /usr/lib/wine/wine.bin ...
Warning: memory above 0x80000000 doesn't seem to be accessible.
Wine requires a 3G/1G user/kernel memory split to work properly.
/usr/bin/wine: line 296: 13289 Killed winecfg
Nowhere to be found
I was also looking for a newer version of 4G/4G patch, even asked Ingo, but all to no avail.
Try 2G/2G split (CONFIG_VMSPLIT_2G) or even 1G/3G (CONFIG_VMSPLIT_1G) if it works for your workload. Also, enable pagetables in highmem (CONFIG_HIGHPTE). That should help a little or might even resolve all your troubles.
Also check memory management tunables if there's something to tweak for your case.
Good luck!
CONFIG_VMSPLIT_2G & CONFIG_VMSPLIT_1G
Are these config options available in 2.6.17 kernel?
I downloaded 2.6.17 upgrade kernel and I could not locate the CONFIG option. Could you please point me how to use 2G/2G or 3G/1G options.
Thanks
Kumar
Embedded
Enable config embedded to find them. It was hidden behind there to make sure noone enabled it without knowing the real consequences of the option.
2G/2SG split (or 3G/1G user/kernel split)
Thanks Con,
It could find it now.
Also, It seems to work like charm for me. But I am bit nervous now.
Situation is like as follows -
My FC4 machine have 2 GB RAM. Eralier it had High Mem enabled and system can see full 2GB memory (1GB low mem and 1GB high mem).
I had a non-root user program that was opening 100's of thosands of sockets.
As the program runs, I can see free low mem keep on reducing and upltimately becomes 0 (while full 1 GB high mem of RAM is still available). Around this time OOM kicks in and start killing some process and system hangs. (A simple user process brough the system to halt!. I was rather expecting new sockets calls to fail and system integrity maintained. Also, I would have expected swapping to kick in before OOM).
After using 2G/2G split, 2 GB is lowmem and 0 highmem. In this case, my socket opening program could create as many sockets and could use whole of 2 GB RAM. Also, once mem reaches around 0, sockets opening starts failing and I also see swapping also kicks in. No OOM business or no system hanging!. All seems great like one one would expect from a solid kernel. I can run a user process malloc'ing several chunks. I could very well alloacte Gig's of mem.
As i understand (I am no kernel expert, just an application programmer making extensive use of system calls), low mem can be used by kernel or user space equally well while high mem can be used by user space programs only. Is my understanding correct? If that is the case, what is the point in having any mem as highmem (if that is possible to configure!).
Since, 2G/2G split seem to be hidden so deep down, i am sure there are some caveats and i am not aware of some dangerous implications.
Could some one explain.
Thanks
Kumar
As i understand (I am no kern
As i understand (I am no kernel expert, just an application programmer making extensive use of system calls), low mem can be used by kernel or user space equally well while high mem can be used by user space programs only. Is my understanding correct?
Yes, lowmem (normal mem) can be used by kernel space or userspace. Highmem can be used in kernelspace, but requires lots of trickery (so not many kernel apps use it). That said.. the 1/3 2/2 3/1 split isn't a split of the real memory, its a split of the address space (that is the ram visible to a process). So for any given process, the addresses in the first 1/2/3 gigs are addresses and can only be used by the kernel, while the rest is userspace. So if you have a 3/1 split, the application cannot mmap()/malloc() more than 1 gig (or 3gigs if you have a 3/1 split).. Also, the kernelspace part of the memory is the same for all processes, while the userspace part is obviously not the same. If that is the case, what is the point in having any mem as highmem (if that is possible to configure!). Well, if you have 2 gigs of ram.. you can use a 2/2 split.. and you wont have any highmem, but then your application is limited to mapping 2 gigs of other stuff. So lets says its an opengl app and you want to map part of the video card memory into you address space (lets say 500 megs) then you can only malloc 1.5 gig because you run out of addresses. If you have a 1/3 split, you app could mmap() 3 gigs so 2 gig could be in physical memory and the rest is in the swap (if its been written to). The easy solution to that problem is to use 64bit addresses. So if you use amd64 or any other 64bit architecture, you dont have that problem. Since you have a very large address space, you can map anything anywhere and you could even map everything everywhere if you wanted...Minor correction
So for any given process, the addresses in the first 1/2/3 gigs are addresses and can only be used by the kernel, while the rest is userspace.
In fact user space come before kernel space.
2/2 vs 3/1
> Since, 2G/2G split seem to be hidden so deep down, i am sure there
> are some caveats and i am not aware of some dangerous implications.
AFAIK, the only downside is that now your userspace programs/processes have slightly less virtual memory available to them.
IIRC, with standard 3/1 split you have something like 2.5GB virtual space available for every process - that is how big virtual memory size any of your processes can reach (by using mmap() or allocating memory from the heap).
When you configure kernel with 2/2 split, available virtual memory per process drops to something like 1.5GB (1GB less).
So it's up to you, if you have any really big/fat processes (typically some databases, Oracle comes to mind) 2/2 won't be a good choice. But in most cases/workloads you wouldn't have such heavy processes, so IMHO in those cases using 2/2 split doesn't have any downside, but only positive effects - simpler, faster and more fair memory management.
wine doesn't like it
> AFAIK, the only downside is that now your userspace programs/processes have slightly less virtual memory available to them.
Oh well, here's the first application I have just encountered that has problems with non-default kernel/user split:
Wine called with no arguments.
Invoking /usr/lib/wine/wine.bin ...
Warning: memory above 0x80000000 doesn't seem to be accessible.
Wine requires a 3G/1G user/kernel memory split to work properly.
/usr/bin/wine: line 296: 13289 Killed winecfg