Re: [PATCH 0/8][for -mm] mem_notify v6

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Tom May
Date: Friday, May 2, 2008 - 3:21 pm

On Thu, May 1, 2008 at 8:06 AM, KOSAKI Motohiro
<kosaki.motohiro@jp.fujitsu.com> wrote:

Some of each.


mlocking didn't fix things, it just made the oom happen at a different
time (see graphs below), both in the small test program where I used
mlockall, and in the jvm where during initialization I read
/proc/self/maps and mlocked each region of memory that was mapped to a
file.  Note that without swap, all of the anonymous pages containing
the java code are effectively locked in memory, too, so everything
runs without page faults.


Most simply, I need to get low memory notifications while there is
still enough memory to handle them before oom.


x86, Linux 2.6.23.9 (with your patches trivially backported), 128MB,
no swap.  Is there something else I can tell you?


It's not what /proc/meminfo calls "Mapped".  It's in anonymous pages
with no backing store, i.e., mmap with MAP_ANONYMOUS.


Here's a graph of MemFree, Cached, and Mapped over time (I believe
Mapped is mostly or entirely subset of Cached here, so it's not
actually important):

http://www.tommay.net/memory.png

The amount of MemFree fluctuates as java allocates and garbage
collects, but the Cached memory decreases (since the kernel has to use
it for __alloc_pages when memory is low) until at some point there is
no memory to satisfy __alloc_pages and there is an oom.

The same things happens if I use mlock, only it happens sooner because
the kernel can't discard any of the 15MB of mlock'd memory so it
actually runs out of memory faster:

http://www.tommay.net/memory-mlock.png

I'm not sure how to explain it differently than I have before.  Maybe
someone else could explain it better.  So, at the risk of merely
repeating myself: The jvm allocates memory.  MemFree decreases.  In
the kernel, __alloc_pages is called.  It finds that memory is low,
memory_pressure_notify is called, and some cached pages are moved to
the inactive list.  These pages may then be used to satisfy
__alloc_pages requests.  The jvm gets the notification, collects
garbage, and returns memory to the kernel which appears as MemFree in
/proc/meminfo.  The cycle continues: the jvm allocates memory until
memory_pressure_notify is called, more cached pages are moved to the
inactive list, etc.  Eventually there are no more pages to move to the
inactive list, and __alloc_pages will invoke the oom killer.


Of course.


My thought was to notify only when the threshold is crossed, i.e.,
edge-triggered not level-triggered.  But I now think a threshold
mechanism may be too complicated, and artificially putting pages in
the cache works just as well.  As a proof-of-concept, I do this, and
it works well, but is inefficient:

   extern char _text;
   for (int i = 0; i < bytes; i += 4096) {
       *((volatile char *)&_text + i);
   }


I think embedded Java is a perfect user of /dev/mem_notify :-) I was
happy to see your patches and the criteria you used for notification.
But I'm having a problem in practice :-(

.tom
--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[PATCH 0/8][for -mm] mem_notify v6, KOSAKI Motohiro, (Sat Feb 9, 8:19 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, Jon Masters, (Sat Feb 9, 9:02 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, KOSAKI Motohiro, (Sat Feb 9, 9:33 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, Rik van Riel, (Sat Feb 9, 9:43 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, KOSAKI Motohiro, (Sat Feb 9, 9:49 am)
[PATCH 0/8][for -mm] mem_notify v6, Jonathan Corbet, (Mon Feb 11, 8:36 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, KOSAKI Motohiro, (Mon Feb 11, 8:46 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, Paul Jackson, (Sun Feb 17, 7:49 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, KOSAKI Motohiro, (Tue Feb 19, 12:36 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, Paul Jackson, (Tue Feb 19, 8:00 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, Rik van Riel, (Tue Feb 19, 12:02 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, Paul Jackson, (Tue Feb 19, 1:18 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, Paul Jackson, (Tue Feb 19, 1:43 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, Pavel Machek, (Tue Feb 19, 3:28 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, Paul Jackson, (Tue Feb 19, 6:54 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, Rik van Riel, (Tue Feb 19, 7:07 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, KOSAKI Motohiro, (Tue Feb 19, 7:48 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, Paul Jackson, (Tue Feb 19, 9:36 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, Paul Jackson, (Tue Feb 19, 9:57 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, KOSAKI Motohiro, (Tue Feb 19, 10:21 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, Tom May, (Tue Apr 1, 4:35 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, KOSAKI Motohiro, (Wed Apr 2, 12:31 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, Tom May, (Wed Apr 2, 10:45 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, Tom May, (Mon Apr 14, 5:16 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, KOSAKI Motohiro, (Tue Apr 15, 7:30 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, KOSAKI Motohiro, (Thu Apr 17, 2:30 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, Tom May, (Thu Apr 17, 12:23 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, KOSAKI Motohiro, (Fri Apr 18, 3:07 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, Tom May, (Mon Apr 21, 1:32 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, Daniel Spång, (Wed Apr 23, 1:27 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, Tom May, (Wed Apr 30, 7:07 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, KOSAKI Motohiro, (Thu May 1, 8:06 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, Tom May, (Fri May 2, 3:21 pm)
Re: [PATCH 0/8][for -mm] mem_notify v6, KOSAKI Motohiro, (Sat May 3, 5:26 am)
Re: [PATCH 0/8][for -mm] mem_notify v6, Tom May, (Mon May 5, 10:22 pm)