Re: Hardcore trashing without any swap

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: László Monda
Date: Friday, June 11, 2010 - 2:38 pm

On Fri, Jun 11, 2010 at 3:47 PM, Ed Tomlinson <edt@aei.ca> wrote:

By reading your reply I think I understood what's going on.

Previously I thought that trashing can only happen because the kernel
discards pages to swap which I thought is the only reason why disk I/O
can happen in such a case.  It didn't made sense to me because I don't
use swap.

The other possibility that occured to me is the following scenario:
1) Process A grows really big and fills up the RAM.
2) The kernel discards code pages from process B in favor of process A.
3) Process A has some RAM and keeps growing further until it almost
fills up the RAM.
4) Process B gets scheduled and needs to be paged in which makes
process A paged out, hence disk I/O occurs.
5) Process A gets scheduled and needs to be paged in which makes
process B paged out, hence disk I/O occurs.
6) Go to 4)

This is not truly an infinite loop because the memory gradually gets
filled up in 4) and 5) but it happens really slowly because process
code has to be paged in upon every rescheduling.

So I've just realized that trashing cannot only happen due to paging
out to swap.  It can also happen due to simply discarding code pages
and paging them in later.

Is the above scenario valid in my case?

I suppose that the OOM killer could kill process A in step 3) right
away but the scheduler is fast and the 4)-5) loop keeps making disk
I/O for a very long time until process A exhausts the memory and the
OOM killer intervenes eventually.

I've also realized that it's probably impossible to truly reliably
foresee trashing so that's why there are no out of the box solutions
for my problem.  The best hack I can think of is a daemon that
monitors memory usage in about every 10 milisec and kills the biggest
process if something seems to go wrong before trashing can begin.


Although I feel that my understanding about this topic is rather
vague, it seems to me that the best settings for me is policy 0 and 0
percentage.  Would this setting avoid trashing with no swap?

-- 
László Monda <http://monda.hu>
--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
Hardcore trashing without any swap, László Monda, (Thu Jun 10, 5:10 pm)
Re: Hardcore trashing without any swap, Alan Cox, (Fri Jun 11, 5:16 am)
Re: Hardcore trashing without any swap, László Monda, (Fri Jun 11, 5:53 am)
Re: Hardcore trashing without any swap, Ed Tomlinson, (Fri Jun 11, 6:47 am)
Re: Hardcore trashing without any swap, László Monda, (Fri Jun 11, 2:38 pm)
Re: Hardcore trashing without any swap, KAMEZAWA Hiroyuki, (Sun Jun 13, 7:12 pm)
Re: Hardcore trashing without any swap, László Monda, (Mon Jun 14, 5:35 am)