Following recent discussions about tuning swapiness [story [1]] and dynamically allocating swap as needed [story [2]], a new discussion ensued on the lkml [3] questioning the need for swap altogether when a system has "sufficient" ram.

Nick Piggin [interview [4]] explained that swap can improve performance no matter how much RAM you have, "well it is a magical property of swap space, because extra RAM doesn't allow you to replace unused memory with often used memory. The theory holds true no matter how much RAM you have. Swap can improve performance. It can be trivially demonstrated." This said, numerous Linux users do report success running a swapless system.

As to why swap performance might be becoming more and more an issue over time, Rik van Riel [interview [5]] suggested that this could be due to the growing gap between the speed of RAM and the speed of hard drives, "the speed of hard disks doesn't grow anywhere near as fast as the size of memory and applications. This means that over the last years, swapping in any particular application has gotten SLOWER than it used to be ... This means that even though the VM is way smarter than it used to be, the visibility of any wrong decision has increased."

From: Anthony DiSante [email blocked]
To:  linux-kernel
Subject: why swap at all?
Date: 	Wed, 26 May 2004 02:38:23 -0400

As a general question about ram/swap and relating to some of the issues in 
this thread:

	~500 megs cached yet 2.6.5 goes into swap hell

Consider this: I have a desktop system with 256MB ram, so I make a 256MB 
swap partition.  So I have 512MB "memory" and if some process wants more, 
too bad, there is no more.

Now I buy another 256MB of ram, so I have 512MB of real memory.  Why not 
just disable my swap completely now?  I won't have increased my memory's 
size at all, but won't I have increased its performance lots?

Or, to make it more appealing, say I initially had 512MB ram and now I have 
1GB.  Wouldn't I much rather not use swap at all anymore, in this case, on 
my desktop?

-Anthony
http://nodivisions.com/ [6]



From: Buddy Lumpkin [email blocked]
Subject: RE: why swap at all?
Date: 	Wed, 26 May 2004 00:31:16 -0700

This is a really good point. I think the bar should be set at max
performance for systems that never need to use the swap device. 

If someone wants to tune swap performance to their hearts content, so be it.
But given cheap prices for memory, and the horrible best case performance
for swap, an increase in swap performance should never, ever come at the
expense of performance for a system that has been sized such that executable
address spaces, libraries and anonymous memory will fit easily within
physical ram.

This of course doesn't address the VM paging storms that happen due to large
amounts of file system writes. Once the pagecache fills up, dirty pages must
be evicted from the pagecache so that new pages can be added to the
pagecache.

--Buddy


From: Nick Piggin [7] [email blocked]
Subject: Re: why swap at all?
Date: 	Wed, 26 May 2004 18:44:58 +1000

Buddy Lumpkin wrote:
> No. I am not making any assertions whatsoever. I am just calling out that
> systems that run happily from physical memory and are not in need of swap
> should never sacrifice an ounce of performance for even drastic improvements
> to swap performance. Swap is a band-aid for saving money on memory and a few
> years ago, it allowed you to save a substantial amount of money. 
> 

Hi Buddy,
Even for systems that don't *need* the extra memory space, swap can
actually provide performance improvements by allowing unused memory
to be replaced with often-used memory.

For example, I have 57MB swapped right now. It allows me to instantly
grep the kernel tree. If I turned swap off, each grep would probably
take 30 seconds.

The VM doesn't always get it right, and to make matters worse, desktop
users don't appreciate their long running jobs finishing earlier, but
*hate* having to wait a few seconds for a window to appear if it hasn't
been used for 24 hours.

> Whether the cost savings for utilizing swap vs buying more memory are
> substantial as of late is subject to opinion, but I cannot think of a system
> that I have sized in the last three years where swap was expected to be used
> except in un-anticipated memory shortfalls. In fact, if I didn't plan to
> store crash dumps on the swap device, I think I would have omitted swap all
> together in many configurations.
> 
> I have worked at large fortune 500 companies with deep pockets though, so
> this may not be the case for many. I make this point though because I think
> if it isn't the case yet, it will be in the near future as memory becomes
> even cheaper because the trend certainly exists.
> 
> As for your short, two sentence comment below, let me save you the energy of
> insinuations and translate your message the way I read it: 
> 

[snip]

I think the comment was rather directed at a specific problem you
described:

 > This of course doesn't address the VM paging storms that happen due to large
 > amounts of file system writes. Once the pagecache fills up, dirty pages must
 > be evicted from the pagecache so that new pages can be added to the
 > pagecache.

This sounds like you are having a serious problem, and it would be
great if you could describe it in detail. kernel version, workload,
description of the system, vmstat output, etc.

Let's keep it nice.

Best regards
Nick



From: John Bradford [email blocked]
Subject: Re: why swap at all?
Date: 	Wed, 26 May 2004 10:34:37 +0100

Quote from Nick Piggin [email blocked]:
> Even for systems that don't *need* the extra memory space, swap can
> actually provide performance improvements by allowing unused memory
> to be replaced with often-used memory.

That's true, but it's not a magical property of swap space - extra physical
RAM would do more or less the same thing.

John.



From: Nick Piggin [8] [email blocked]
Subject: Re: why swap at all?
Date: 	Wed, 26 May 2004 19:48:10 +1000

John Bradford wrote:
> Quote from Nick Piggin [email blocked]:
> 
>>Even for systems that don't *need* the extra memory space, swap can
>>actually provide performance improvements by allowing unused memory
>>to be replaced with often-used memory.
> 
> 
> That's true, but it's not a magical property of swap space - extra physical
> RAM would do more or less the same thing.
> 

Well it is a magical property of swap space, because extra RAM
doesn't allow you to replace unused memory with often used memory.

The theory holds true no matter how much RAM you have. Swap can
improve performance. It can be trivially demonstrated.


From: Martin Olsson [email blocked]
Subject: Re: why swap at all?
Date: 	Wed, 26 May 2004 12:45:26 +0200

Hi Linux-gurus,

I agree with Anthony Disante, maybe not all users want swapping. I have 
myself felt very annoying with swapping lately but I've not yet tried to 
disable it.

In school I've studied the swapping concept from a theoretical point
of view, and I fully understand the fact that swapping, if used 
properly, can both increase performance and provide a safe way to get 
out of a bad situation when the box runs out of memory. The problem is 
that in reality this does not work, not on Linux nor on Windows 2000 
which I use at home. Unfortunately I cannot provide a specific reason 
why it does not work, I'm very much a end-user/desktop-user, I'm not a 
kernel hacker (yet). But I see two things that needs improvement atm:

A) when I do large data processing operations the computer is always 
very very slow afterwards

B) if I have X Mb of RAM then there should not be imho a single swap 
read/write until the whole of my X Mb RAM is completely stuffed, is this 
so today?

---

Also, imagine that I disable swap today and start a large data 
processing operation. During this operation I try to start a new 
process, here ideally the program should not OOM but instead the memory 
allocated for the data processing operation should be decreased. Is this 
possible using today's technology? Can be divide memory into two sorts, 
one for processes (here to stay memory) and another sort for batch 
operations (where the amount of memory does not really matter but less 
memory means less performance). I see the problem with "taking memory 
back" though, I guess its impossible.



Regards,
/m



From: David Schwartz [email blocked]
Subject: RE: why swap at all?
Date: 	Wed, 26 May 2004 09:33:44 -0700


> I agree with Anthony Disante, maybe not all users want swapping. I have
> myself felt very annoying with swapping lately but I've not yet tried to
> disable it.

	You're probably really just annoyed with physical memory that's too small
to hold your working set. Believe it or not, having swap delays the onset of
this problem.

> In school I've studied the swapping concept from a theoretical point
> of view, and I fully understand the fact that swapping, if used
> properly, can both increase performance and provide a safe way to get
> out of a bad situation when the box runs out of memory. The problem is
> that in reality this does not work, not on Linux nor on Windows 2000
> which I use at home. Unfortunately I cannot provide a specific reason
> why it does not work, I'm very much a end-user/desktop-user, I'm not a
> kernel hacker (yet). But I see two things that needs improvement atm:

	I don't think you really do understand it from a theoretical point of view,
because you say:

> A) when I do large data processing operations the computer is always
> very very slow afterwards

	Of course, this is because the working set has changed. However, with swap,
the least used pages can be evicted from physical memory. Without it, there
may be no place to put the least used pages and more frequently used pages
have to be evicted.

> B) if I have X Mb of RAM then there should not be imho a single swap
> read/write until the whole of my X Mb RAM is completely stuffed, is this
> so today?

	It depends what you mean by "stuffed". On a modern operating system like
Linux, pretty much all of your physical memory is in use all the time.
Without swap, dirty pages cannot be evicted from physical memory, even if
they haven't been used for days. If your physical memory exceeds your
working set size, you win no matter what. But without swap, every dirty page
is part of your working set, even if it hasn't been read/written for days.

> Also, imagine that I disable swap today and start a large data
> processing operation. During this operation I try to start a new
> process, here ideally the program should not OOM but instead the memory
> allocated for the data processing operation should be decreased. Is this
> possible using today's technology? Can be divide memory into two sorts,
> one for processes (here to stay memory) and another sort for batch
> operations (where the amount of memory does not really matter but less
> memory means less performance). I see the problem with "taking memory
> back" though, I guess its impossible.

	No, it's not difficult. The OS takes physical memory back all the time by
swapping.

	You seem to be missing a fundamental concept. Physical memory will always
get full because the OS will always keep copies of file data in memory just
in case it needs them again. Because new pages are always being read in and
processes are always allocating new memory, the OS will have to make a
decision of what pages to evict from physical memory. If a page is dirty, it
can only be evicted if there's swap. So if you have dirty pages that are
very rarely used, swap allows you to keep more hot, clean pages in memory.

	A lot of people feel subjectively that swap makes a system slow. There's
anecdotal evidence that swap does horrible things or "must be badly broken
because the machine gets slow" on almost every operating system that
supports swapping. In most cases, it's just a case where the real working
set has exceeded physical memory, and in that case, swap is just doing what
it's supposed to be doing.

	DS

From: Denis Vlasenko [email blocked]
Subject: RE: why swap at all?
Date: 	Wed, 26 May 2004 13:41:10 +0300

On Wednesday 26 May 2004 11:30, Buddy Lumpkin wrote:
> I have worked at large fortune 500 companies with deep pockets though, so
> this may not be the case for many. I make this point though because I think
> if it isn't the case yet, it will be in the near future as memory becomes
> even cheaper because the trend certainly exists.

"640k will be enough for anyone" ?

No. Unfortunately, userspace programs grow in size as fast
as your RAM. Because typically developers do not think
about size of their program until it starts to outgrow
their RAM.

Today, 128M RAM swapless is barely enough to run full
spectrum of apps. OpenOffice and Mozilla "lead" the pack,
followed by KDE/Gnome etc.
-- 
vda




From: Rik van Riel [9] [email blocked]
Subject: RE: why swap at all?


Date: 	Wed, 26 May 2004 08:30:59 -0400 (EDT)

On Wed, 26 May 2004, Denis Vlasenko wrote:

> No. Unfortunately, userspace programs grow in size as fast
> as your RAM. Because typically developers do not think
> about size of their program until it starts to outgrow
> their RAM.

It's worse than that.  Way worse.

The speed of hard disks doesn't grow anywhere near as
fast as the size of memory and applications. This means
that over the last years, swapping in any particular
application has gotten SLOWER than it used to be ...

This means that even though the VM is way smarter than
it used to be, the visibility of any wrong decision has
increased.

I wonder if there's a way we could change the VM so it
could recover faster from any mistakes it made, instead
of trying to prevent it from making any mistakes (those
will happen anyway, the VM can't predict the future).

-- 
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan


From: Rik van Riel [10] [email blocked]
Subject: RE: why swap at all?
Date: 	Wed, 26 May 2004 08:19:34 -0400 (EDT)

On Wed, 26 May 2004, Buddy Lumpkin wrote:

> No. I am not making any assertions whatsoever. I am just calling out
> that systems that run happily from physical memory and are not in need
> of swap should never sacrifice an ounce of performance

Executables and shared libraries live in the filesystem
cache.  Evicting those from memory - because swapping is
disabled and "the VM should remove something from cache
instead" - will feel exactly the same as swapping ...

-- 
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan


From: Roger Luethi [email blocked]
Subject: Re: why swap at all?
Date: 	Wed, 26 May 2004 10:27:12 +0200

On Wed, 26 May 2004 02:38:23 -0400, Anthony DiSante wrote:
> Now I buy another 256MB of ram, so I have 512MB of real memory.  Why not 
> just disable my swap completely now?  I won't have increased my memory's 
> size at all, but won't I have increased its performance lots?
> 
> Or, to make it more appealing, say I initially had 512MB ram and now I have 
> 1GB.  Wouldn't I much rather not use swap at all anymore, in this case, on 
> my desktop?

Swap serves another (often underrated) purpose: Graceful degradation.

If you have a reasonably amount of swap space mounted, you will know
you are running out of RAM because your system will become noticeably
slower. If you have no swap whatsoever, your first warning will quite
possibly be an application OOM killed or losing data due to a failed
memory allocation.

Think of the slowness of swap as a _feature_.

Roger



From: John Bradford [email blocked]
Subject: Re: why swap at all?
Date: 	Wed, 26 May 2004 10:23:32 +0100

Quote from Roger Luethi [email blocked]:
> On Wed, 26 May 2004 02:38:23 -0400, Anthony DiSante wrote:
> > Now I buy another 256MB of ram, so I have 512MB of real memory.  Why not 
> > just disable my swap completely now?  I won't have increased my memory's 
> > size at all, but won't I have increased its performance lots?
> > 
> > Or, to make it more appealing, say I initially had 512MB ram and now I have 
> > 1GB.  Wouldn't I much rather not use swap at all anymore, in this case, on 
> > my desktop?
> 
> Swap serves another (often underrated) purpose: Graceful degradation.
> 
> If you have a reasonably amount of swap space mounted, you will know
> you are running out of RAM because your system will become noticeably
> slower. If you have no swap whatsoever, your first warning will quite
> possibly be an application OOM killed or losing data due to a failed
> memory allocation.
> 
> Think of the slowness of swap as a _feature_.

There is a very negative side to this approach as well, especially if users
allocate excessive swap space.

A run-away process on a server with too much swap can cause it to grind to
almost a complete halt, and become almost compltely unresponsive to remote
connections.

If the total amount of storage is just enough for the tasks the server is
expected to deal with, then a run-away process will likely be terminated
quickly stopping it from causing the machine to grind to a halt.

If, on the other hand, there is excessive storage, it can continue running
for a long time, often consuming a lot of CPU.

When the excess storage is physical RAM, this might not be particularly
disasterous, but if it's swap space, it's much more likely to cause a serious
drop in performance.

For a desktop system, it might not be a big deal, but when it's an ISP's server
in a remote data centre, it can create a lot of unnecessary work.

John.



From: Roger Luethi [email blocked]
Subject: Re: why swap at all?
Date: 	Wed, 26 May 2004 11:30:08 +0200

On Wed, 26 May 2004 10:23:32 +0100, John Bradford wrote:
> A run-away process on a server with too much swap can cause it to grind to
> almost a complete halt, and become almost compltely unresponsive to remote
> connections.
> 
> If the total amount of storage is just enough for the tasks the server is
> expected to deal with, then a run-away process will likely be terminated
> quickly stopping it from causing the machine to grind to a halt.

I'm not sure your optimism about the correct (run-away) process being
terminated is justified. Granted, there are definitely scenarios
where swapless operation is preferable, but in most circumstances --
especially workstations as the original poster described -- I'd rather
minimize the risk of losing data.

Roger



From: John Bradford [email blocked]
Subject: Re: why swap at all?
Date: 	Wed, 26 May 2004 11:35:53 +0100

Quote from Roger Luethi [email blocked]:
> On Wed, 26 May 2004 10:23:32 +0100, John Bradford wrote:
> > A run-away process on a server with too much swap can cause it to grind to
> > almost a complete halt, and become almost compltely unresponsive to remote
> > connections.
> > 
> > If the total amount of storage is just enough for the tasks the server is
> > expected to deal with, then a run-away process will likely be terminated
> > quickly stopping it from causing the machine to grind to a halt.
> 
> I'm not sure your optimism about the correct (run-away) process being
> terminated is justified. Granted, there are definitely scenarios
> where swapless operation is preferable, but in most circumstances --
> especially workstations as the original poster described -- I'd rather
> minimize the risk of losing data.

Well, I am basing this on experience.  I know an ISP who had their main
customer webserver down for hours because of this kind of problem - the whole
thing created a lot of work and wasted a lot of time.

In this particular scenario, I think the run-away process was probably using
up almost two thirds of the total RAM, so I'm pretty confident the correct
process would have been terminated.

I know that trusting the kernel to terminate the correct run-away process
might seem like a bit of a risky approach to take with respect to
loosing data, especially where a little bit of swap space might come to the
rescue.

However, in my opinion, if a machine has insufficient storage for the intended
task then that's an error condition straight away.  So, I am not really
concerned with trying to make sure that a desktop system running an
application which the user has underestimated the memory usage of doesn't crash
no matter what.  The machine is operating in an error condition, so data loss
should be expected.

No, I am more concerned about preventing unexpected usage of a machine from
causing large scale slowdowns, and unavailability to other users.

For example, if a run-away process occurs, or one user on a multi-user system
uses up excessive resources.

Excessive swap space might create an illusion of protecting against data loss,
by allowing things to continue working no matter what, just a bit slower, but
for multi user systems, it's preventing normal usage of the system.  This can
indirectly lead to data loss if the machine is not accessible over the network
to perform a critical function.

Ultimately, once a machine is spending 99% of it's time swapping, it's likely
to be well past the point where it's practical to log in remotely and fix it.

However, I think that there are probably more machines using excessive swap
which would benefit from reducing it, than the other way round, though, simply
because users are not as aware of the potential problems.

My opinion was that the machine was already in an error condition the minute
I couldn't access it remotely - a significant number of customer's webpages
were inaccessible, which potentially means lost business for them.

I assume that the scenario you were thinking of when you mentioned data loss
above was a system running a critical process which is using, for example,
90% of the available storage - in that case if another process starts up, and
uses up the rest of the available storage, then the first process will probably
be terminated, whereas if you increase the amount of storage, (either by adding
swap or physical RAM), then the second process can continue for longer.

However, in this situation, I can see two possibilities - either the second
process is genuine, (I.E. not a run-away process), in which case the machine
has insufficient storage for it's intended purpose, which is an operator error
in my opinion, or the process is a run-away process, in which case a little
extra storage isn't going to do much other than buy time before the first
process is terminated.  This may give an operator chance to log in and fix
the problem, (probably by terminating the run-away process), but if this extra
storage is swap space, the machine may well become unresponsive very quickly,
making it virtually impossible to log in remotely, and making other network
services on that machine virtually inaccessible.  Eventually the run-away
process may use up the swap space, and then the first process will probably
be terminated as before, just not as quickly.  If instead of a little extra
storage, a lot was added such that the first process was no longer using more
storage than the run-away process was when storage was full, then the kernel
will hopefully terminate the run-away process, but probably only after the
machine has been unresponsive for a long time, possibly causing other problems.

Basically, I would be skeptical about using a desktop system where one
terminated process could cause data loss to the extent that I couldn't easily
restore the data.

John.



From: Nick Piggin [11] [email blocked]
Subject: Re: why swap at all?
Date: 	Wed, 26 May 2004 20:37:50 +1000

John Bradford wrote:
> 
> 
> Well, I am basing this on experience.  I know an ISP who had their main
> customer webserver down for hours because of this kind of problem - the whole
> thing created a lot of work and wasted a lot of time.
> 
> In this particular scenario, I think the run-away process was probably using
> up almost two thirds of the total RAM, so I'm pretty confident the correct
> process would have been terminated.
> 

I think this is somewhat orthogonal to whether swap should be
used or not.

What we should be doing here is enforcing the RSS rlimit. I
have a patch from Rik to do this which needs to be merged.

Hopefully this would give you the best case situation of
having only the runaway process really slow down, without
killing anything until the admin arrives.

• Archive of above thread [Array]
• KernelTrap interview with William Lee Irwin [12]
• KernelTrap interview with Nick Piggin [13]
• KernelTrap interview with Rik van Riel [14]