On Thu, May 1, 2008 at 8:06 AM, KOSAKI Motohiro
<kosaki.motohiro@jp.fujitsu.com> wrote:
Some of each.
mlocking didn't fix things, it just made the oom happen at a different
time (see graphs below), both in the small test program where I used
mlockall, and in the jvm where during initialization I read
/proc/self/maps and mlocked each region of memory that was mapped to a
file. Note that without swap, all of the anonymous pages containing
the java code are effectively locked in memory, too, so everything
runs without page faults.
Most simply, I need to get low memory notifications while there is
still enough memory to handle them before oom.
x86, Linux 2.6.23.9 (with your patches trivially backported), 128MB,
no swap. Is there something else I can tell you?
It's not what /proc/meminfo calls "Mapped". It's in anonymous pages
with no backing store, i.e., mmap with MAP_ANONYMOUS.
Here's a graph of MemFree, Cached, and Mapped over time (I believe
Mapped is mostly or entirely subset of Cached here, so it's not
actually important):
http://www.tommay.net/memory.png
The amount of MemFree fluctuates as java allocates and garbage
collects, but the Cached memory decreases (since the kernel has to use
it for __alloc_pages when memory is low) until at some point there is
no memory to satisfy __alloc_pages and there is an oom.
The same things happens if I use mlock, only it happens sooner because
the kernel can't discard any of the 15MB of mlock'd memory so it
actually runs out of memory faster:
http://www.tommay.net/memory-mlock.png
I'm not sure how to explain it differently than I have before. Maybe
someone else could explain it better. So, at the risk of merely
repeating myself: The jvm allocates memory. MemFree decreases. In
the kernel, __alloc_pages is called. It finds that memory is low,
memory_pressure_notify is called, and some cached pages are moved to
the inactive list. These pages may then be used to satisfy
__alloc_pages requests. The jvm gets the notification, collects
garbage, and returns memory to the kernel which appears as MemFree in
/proc/meminfo. The cycle continues: the jvm allocates memory until
memory_pressure_notify is called, more cached pages are moved to the
inactive list, etc. Eventually there are no more pages to move to the
inactive list, and __alloc_pages will invoke the oom killer.
Of course.
My thought was to notify only when the threshold is crossed, i.e.,
edge-triggered not level-triggered. But I now think a threshold
mechanism may be too complicated, and artificially putting pages in
the cache works just as well. As a proof-of-concept, I do this, and
it works well, but is inefficient:
extern char _text;
for (int i = 0; i < bytes; i += 4096) {
*((volatile char *)&_text + i);
}
I think embedded Java is a perfect user of /dev/mem_notify :-) I was
happy to see your patches and the criteria you used for notification.
But I'm having a problem in practice :-(
.tom
--
