I'm not sure how helpful this is. It sounds like you may be missing
something, but I'm not competent to explain it. But here are some of
my thoughts.
1. If your system is really running away, maybe killing the processes
responsible is a good idea.
2. Why do you want to set an _overcommit_ soft-limit at 90% of RAM?
Even without swap, that sounds very restrictrive.
If I naively add up the top 10 memory hogs on my 512M system, they're
using a total of over 4G of virtual memory. (And I have less than
512M swap). Some of that will be shared memory or backed by disk
files - but I don't think that it can account for all 3G of the
overcommit. Shared memory tends to be code only, and my entire "disk"
is only 4G.
In other words, I reckon I have on the order of a gigabyte of virtual
address space, which has been malloc'ed or equivalent, but is not used
and therefore requires no memory resource (ram or swap).
3. Personally, without knowing whether it's already done, I think a
significant solution for the desktop is for browsers, mail clients and
similar to place strict limits on their anonymous memory consumption,
and use mmaped files for the caches / object stores which can grow so
large. Similarly, the GIMP uses a "tile cache" file (which perhaps
shows its age). If applications are storing caches on disk, they
should make this clear to the kernel - using mmapped files for cache
effectively turns them into swap files, letting the kernel page them
out to disk without using up the kernel swap file. Exposing the cache
memory usage on disk also makes it slightly more accessible, e.g. to
disk quotas, or users with "du".
--
