[PATCH] exec argument expansion can inappropriately trigger OOM-killer

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Kees Cook
Date: Friday, August 27, 2010 - 3:02 pm

Brad Spengler published a local memory-allocation DoS that
evades the OOM-killer (though not the virtual memory RLIMIT):
http://www.grsecurity.net/~spender/64bit_dos.c

The recent changes to create a stack guard page helps slightly to
discourage this attack, but it is not sufficient. Compiling it statically
moves the libraries out of the way, allowing the stack VMA to fill the
entire TASK_SIZE.

There are two issues:
 1) the OOM killer doesn't notice this argv memory explosion
 2) the argv expansion does not check if rlim[RLIMIT_STACK].rlim_cur is -1.

I figure a quick solution for #2 would be the following patch. However,
running multiple copies of this program could result in similar OOM
behavior, so issue #1 still needs a solution.

Reported-by: Brad Spengler <spender@grsecurity.net>
Signed-off-by: Kees Cook <kees.cook@canonical.com>
---
 fs/exec.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/fs/exec.c b/fs/exec.c
index dab85ec..be40063 100644
--- a/fs/exec.c
+++ b/fs/exec.c
@@ -194,7 +194,8 @@ static struct page *get_arg_page(struct linux_binprm *bprm, unsigned long pos,
 		 *    to work from.
 		 */
 		rlim = current->signal->rlim;
-		if (size > ACCESS_ONCE(rlim[RLIMIT_STACK].rlim_cur) / 4) {
+		if (size > ACCESS_ONCE(rlim[RLIMIT_STACK].rlim_cur) / 4 ||
+		    size > TASK_SIZE / 4) {
 			put_page(page);
 			return NULL;
 		}
-- 
1.7.1

-- 
Kees Cook
Ubuntu Security Team
--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[PATCH] exec argument expansion can inappropriately trigge ..., Kees Cook, (Fri Aug 27, 3:02 pm)
[PATCH 0/3] execve argument-copying fixes, Roland McGrath, (Tue Sep 7, 7:34 pm)
Re: [PATCH 0/3] execve argument-copying fixes, KOSAKI Motohiro, (Tue Sep 7, 8:00 pm)
[PATCH 0/2] execve memory exhaust of argument-copying fixes, KOSAKI Motohiro, (Wed Sep 8, 10:01 pm)
[PATCH 1/2] oom: don't ignore rss in nascent mm, KOSAKI Motohiro, (Wed Sep 8, 10:03 pm)
Re: [PATCH 1/2] oom: don't ignore rss in nascent mm, Oleg Nesterov, (Thu Sep 9, 3:05 pm)
Re: [PATCH 1/2] oom: don't ignore rss in nascent mm, Roland McGrath, (Fri Sep 10, 2:39 am)
Re: [PATCH 2/2] execve: check the VM has enough memory at ..., KOSAKI Motohiro, (Wed Sep 15, 10:51 pm)