OOM notifications

Previous thread: OOM notifications by Marcelo Tosatti on Thursday, October 18, 2007 - 4:15 pm. (12 messages)

Next thread: [git patch] another libata fix by Jeff Garzik on Thursday, October 18, 2007 - 4:24 pm. (2 messages)
To: <linux-kernel@...>
Cc: <riel@...>, <drepper@...>
Date: Thursday, October 18, 2007 - 4:25 pm

Hi,

AIX contains the SIGDANGER signal to notify applications to free up some
unused cached memory:

http://www.ussg.iu.edu/hypermail/linux/kernel/0007.0/0901.html

There have been a few discussions on implementing such an idea on Linux,
but nothing concrete has been achieved.

On the kernel side Rik suggested two notification points: "about to
swap" (for desktop scenarios) and "about to OOM" (for embedded-like
scenarios).

With that assumption in mind it would be necessary to either have two
special devices for notification, or somehow indicate both events
through the same file descriptor.

Comments are more than welcome.

-

To: Marcelo Tosatti <marcelo@...>
Cc: <linux-kernel@...>, <riel@...>, <drepper@...>
Date: Thursday, October 18, 2007 - 4:38 pm

Given the desktop/embedded distinction you made, do you need both scenarios
active at the same time? If not, it seems something like a

echo -n <level> >/proc/sys/vm/danger

could do with just one sigdanger notification point? (with <level> suitably
defined as or in terms of the used threshold value).

Rene.
-

To: Rene Herman <rene.herman@...>
Cc: Marcelo Tosatti <marcelo@...>, <linux-kernel@...>, <drepper@...>
Date: Thursday, October 18, 2007 - 4:52 pm

On Thu, 18 Oct 2007 22:38:21 +0200

If you do that, how are applications to know which of the two
scenarios is happening when they get a signal?

--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan
-

To: Rik van Riel <riel@...>
Cc: Marcelo Tosatti <marcelo@...>, <linux-kernel@...>, <drepper@...>
Date: Thursday, October 18, 2007 - 5:06 pm

They don't -- that's why I asked if you need both scenario's active at the
same time. SIGDANGER would just be SIGPLEASEFREEALLYOUCAN with the operator
deciding through setting the level at which point applications get it.

Or put differently; what's the additional value of notifying an application
that the system is about to go balistic when you've already asked it to free
all it could earlier? SIGSEEDAMNITITOLDYOUSO?

Don't get me wrong; never saw this discussion earlier, may be sensible...

Rene.

-

To: Rene Herman <rene.herman@...>
Cc: Marcelo Tosatti <marcelo@...>, <linux-kernel@...>, <drepper@...>
Date: Thursday, October 18, 2007 - 5:18 pm

On Thu, 18 Oct 2007 23:06:52 +0200

The first threshold - "we are about to swap" - means the application
frees memory that it can. Eg. free()d memory that glibc has not yet
given back to the kernel, or JVM running the garbage collector, or ...

The second threshold - "we are out of memory" - means that the first
approach has failed and the system needs to do something else. On an
embedded system, I would expect some application to exit or maybe
restart itself.

--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan
-

To: Rik van Riel <riel@...>
Cc: Marcelo Tosatti <marcelo@...>, <linux-kernel@...>, <drepper@...>
Date: Thursday, October 18, 2007 - 6:01 pm

That first threshold sounds fine yes. To me, the second mostly sounds like a
job for SIGTERM though.

The OOM killer could after it selected the task for killing first try a TERM
on it to give a chance to exit gracefully and only when that doesn't help
make it eligible for killing on a second round through the badness calculation.

You could moreover _never_ make a task eligible for killing before it
received a SIGTERM, thereby guaranteeing that everyone got the SIGTERM
before killing anything, and it seems SIGTERM would be a more focussed
version of SIGDANGER2 then.

Would at least forego any need for multiplexing the DANGER signal.

Rene.

-

To: Rik van Riel <riel@...>
Cc: Marcelo Tosatti <marcelo@...>, <linux-kernel@...>, <drepper@...>
Date: Thursday, October 18, 2007 - 6:16 pm

Well, no, that "guarantee" is fairly badly formulated but I mean "before
everyone got a SIGTERM" ofcourse. That is, first do the same selection as
now but don't send KILL but TERM and mark the task as having received a TERM
already and make it not eligible anymore. Only when there are no TERM

Rene.
-

To: Rene Herman <rene.herman@...>
Cc: Rik van Riel <riel@...>, Marcelo Tosatti <marcelo@...>, <linux-kernel@...>
Date: Thursday, October 18, 2007 - 6:10 pm

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I agree. Applications shouldn't be expected to be yet more complicated
and have different levels of low memory handling. You might want to
give a process a second shot at handling SIGDANGER but after that's it's
all about preparation for a shutdown.

- --
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQFHF9m/2ijCOnn/RHQRAhwjAKC38y1OLv0mE5sWHY31CwJ2ZaoAXwCglDTO
05pmpe8jMVhwM0nlCHqZyaQ=
=5DvG
-----END PGP SIGNATURE-----
-

To: Ulrich Drepper <drepper@...>
Cc: Rene Herman <rene.herman@...>, Rik van Riel <riel@...>, Marcelo Tosatti <marcelo@...>, <linux-kernel@...>
Date: Friday, October 19, 2007 - 6:17 am

That works okay on a PC, but try cellphone one day.

You want management app to close the least used application. You do
not want _kernel_ to select "who to send SIGTERM to".
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-

To: Ulrich Drepper <drepper@...>
Cc: Rene Herman <rene.herman@...>, Rik van Riel <riel@...>, Marcelo Tosatti <marcelo@...>, <linux-kernel@...>
Date: Friday, October 19, 2007 - 1:15 am

I disagree. From an embedded viewpoint it would be nice to have a
"please free up memory", then a "we really need memory NOW", then
finally the kernel oom killer.

The advantage of the middle message is that it allows userspace to do
smarter things if it wants to (for instance, if there is an overall
system manager or some such thing, it could do a better job of
restarting tasks than the kernel oom killer since it knows the relative
importance of tasks).

Chris
-

Previous thread: OOM notifications by Marcelo Tosatti on Thursday, October 18, 2007 - 4:15 pm. (12 messages)

Next thread: [git patch] another libata fix by Jeff Garzik on Thursday, October 18, 2007 - 4:24 pm. (2 messages)