Re: sbrk(2) broken

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Igor Mozolevsky <igor@...>
Cc: Kostik Belousov <kostikbel@...>, Andrew Reilly <andrew-freebsd@...>, <freebsd-current@...>, Peter Jeremy <peterjeremy@...>
Date: Monday, January 7, 2008 - 8:17 pm

In message <a2b6592c0801071606g4c0dcb9ap117e345fda5e7e5f@mail.gmail.com>, "Igor
 Mozolevsky" writes:

You should read some recent literature on malloc(3), my own and
Jasons papers are good places to start.

For performance reasons, malloc(3) will hold on to a number of pages
that theoretically could be given back to the kernel, simply because
it expects to need them shortly.

Such parameters and many others of the malloc implementation can
be tweaked to "waste" more or less memory, in response to a sensibly
granular indication from the kernel about how bad things are.

Also, many subsystems in the kernel could adjust their memory use
in response to a "memory pressure" indication, if memory is tight,
we could cache vnodes and inodes less agressively, if things are
going truly bad, we can even ditch all non-active entries from
these caches.

If one implements this with three states:

Green - "all clear"

Yellow - "tight" - free one before you allocate one if you can.

Red - "all out" - free all that you sensibly can.

And implemented strategies like I propose above (and have proposed
for the last 10 years), then it is very unlikely that the system
would ever get into the red state, because the yellow state will
mitigate and reduce the memory pressure.

Nothing prevents an intelligent process from listening in and
doing sensible things, firefox could ditch the memory cache of
pages for instance.

But we can't get anywhere until some VM wizard produces the
three "lamps" for us to look at in the first place, that's where
we have been stuck for the last 10 years.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
sbrk(2) broken, Jason Evans, (Thu Jan 3, 2:38 am)
Re: sbrk(2) broken, Andrey Chernov, (Fri Jan 4, 8:21 am)
Re: sbrk(2) broken , Poul-Henning Kamp, (Fri Jan 4, 8:57 am)
Re: sbrk(2) broken, Andrey Chernov, (Fri Jan 4, 9:12 am)
Re: sbrk(2) broken, David Taylor, (Fri Jan 4, 9:25 am)
Re: sbrk(2) broken, Andrey Chernov, (Fri Jan 4, 10:22 am)
Re: sbrk(2) broken , Poul-Henning Kamp, (Fri Jan 4, 9:28 am)
Re: sbrk(2) broken, Kostik Belousov, (Thu Jan 3, 4:39 pm)
Re: sbrk(2) broken, Dag-Erling Smørgrav, (Thu Jan 3, 3:21 pm)
Re: sbrk(2) broken, Robert Watson, (Thu Jan 3, 8:26 pm)
Re: sbrk(2) broken, Igor Mozolevsky, (Fri Jan 4, 6:41 am)
Re: sbrk(2) broken, Robert Watson, (Fri Jan 4, 7:19 am)
Re: sbrk(2) broken, Dag-Erling Smørgrav, (Fri Jan 4, 6:55 am)
Re: sbrk(2) broken, Igor Mozolevsky, (Fri Jan 4, 7:18 am)
Re: sbrk(2) broken, Giorgos Keramidas, (Fri Jan 4, 11:26 pm)
Re: sbrk(2) broken, Dag-Erling Smørgrav, (Fri Jan 4, 8:45 am)
Re: sbrk(2) broken , Poul-Henning Kamp, (Fri Jan 4, 8:53 am)
Re: sbrk(2) broken, Igor Mozolevsky, (Fri Jan 4, 9:03 am)
Re: sbrk(2) broken, Dag-Erling Smørgrav, (Fri Jan 4, 9:12 am)
Re: sbrk(2) broken, Kostik Belousov, (Fri Jan 4, 9:48 am)
Re: sbrk(2) broken , Poul-Henning Kamp, (Mon Jan 7, 5:08 am)
Re: sbrk(2) broken, Peter Jeremy, (Mon Jan 7, 5:58 am)
Re: sbrk(2) broken , Poul-Henning Kamp, (Mon Jan 7, 6:05 am)
Re: sbrk(2) broken, Igor Mozolevsky, (Mon Jan 7, 9:15 am)
Re: sbrk(2) broken , Poul-Henning Kamp, (Mon Jan 7, 9:18 am)
Re: sbrk(2) broken, Andrew Reilly, (Mon Jan 7, 7:19 pm)
Re: sbrk(2) broken, Igor Mozolevsky, (Mon Jan 7, 8:06 pm)
Re: sbrk(2) broken , Poul-Henning Kamp, (Mon Jan 7, 8:17 pm)
Re: sbrk(2) broken, Peter Schuller, (Mon Jan 7, 9:37 pm)
Re: sbrk(2) broken, Dag-Erling Smørgrav, (Tue Jan 8, 2:36 pm)
Re: sbrk(2) broken, Peter Schuller, (Wed Jan 9, 2:22 pm)
Re: sbrk(2) broken, Dag-Erling Smørgrav, (Thu Jan 10, 6:04 am)
Re: sbrk(2) broken, Dag-Erling Smørgrav, (Thu Jan 10, 10:31 am)
Re: sbrk(2) broken, Igor Mozolevsky, (Mon Jan 7, 8:57 pm)
Re: sbrk(2) broken , Poul-Henning Kamp, (Tue Jan 8, 4:31 am)
Re: sbrk(2) broken, Alexander Kabaev, (Mon Jan 7, 10:34 pm)
Re: sbrk(2) broken, Andrew Reilly, (Mon Jan 7, 8:28 pm)
Re: sbrk(2) broken, Robert Watson, (Fri Jan 4, 9:24 am)
Re: sbrk(2) broken, Dag-Erling Smørgrav, (Sat Jan 5, 9:50 am)
Re: sbrk(2) broken, Kris Kennaway, (Fri Jan 4, 7:31 am)
Re: sbrk(2) broken, Robert Watson, (Fri Jan 4, 7:22 am)
Re: sbrk(2) broken, Igor Mozolevsky, (Fri Jan 4, 7:30 am)
Re: sbrk(2) broken, Robert Watson, (Fri Jan 4, 7:38 am)
Re: sbrk(2) broken, Dag-Erling Smørgrav, (Fri Jan 4, 8:48 am)
Re: sbrk(2) broken, Dag-Erling Smørgrav, (Fri Jan 4, 5:32 am)
Re: sbrk(2) broken, Robert Watson, (Fri Jan 4, 7:06 am)
Re: sbrk(2) broken, Skip Ford, (Fri Jan 4, 9:54 am)
Re: sbrk(2) broken, Kostik Belousov, (Fri Jan 4, 9:59 am)
Re: sbrk(2) broken, Skip Ford, (Fri Jan 4, 10:11 am)
Re: sbrk(2) broken, Kostik Belousov, (Fri Jan 4, 10:18 am)
Re: sbrk(2) broken, Skip Ford, (Fri Jan 4, 10:58 am)
Re: sbrk(2) broken, Dag-Erling Smørgrav, (Sat Jan 5, 10:01 am)
Re: sbrk(2) broken, Dag-Erling Smørgrav, (Fri Jan 4, 8:34 am)
Re: sbrk(2) broken, Robert Watson, (Fri Jan 4, 9:26 am)
Re: sbrk(2) broken , Ian FREISLICH, (Fri Jan 4, 2:27 am)
Re: sbrk(2) broken, Peter Jeremy, (Fri Jan 4, 5:51 am)
Re: sbrk(2) broken, Kostik Belousov, (Fri Jan 4, 8:47 am)
Re: sbrk(2) broken, Scott Long, (Thu Jan 3, 6:23 pm)
Re: sbrk(2) broken, John Baldwin, (Thu Jan 3, 6:46 pm)
Re: sbrk(2) broken, Scott Long, (Thu Jan 3, 7:08 pm)
Re: sbrk(2) broken, Robert Watson, (Thu Jan 3, 8:31 pm)
Re: sbrk(2) broken, Peter Schuller, (Thu Jan 3, 5:00 pm)
Re: sbrk(2) broken, Jason Fesler, (Thu Jan 3, 5:08 pm)
Re: sbrk(2) broken, Dag-Erling Smørgrav, (Fri Jan 4, 5:07 am)
Re: sbrk(2) broken, Tim Kientzle, (Fri Jan 4, 1:55 pm)
ELF dynamic loader name [was: sbrk(2) broken], Maxim Sobolev, (Fri Jan 4, 5:25 pm)
Re: ELF dynamic loader name [was: sbrk(2) broken], Peter Wemm, (Fri Jan 4, 5:42 pm)
Re: ELF dynamic loader name [was: sbrk(2) broken], Tim Kientzle, (Fri Jan 4, 11:51 pm)
Re: ELF dynamic loader name, Dag-Erling Smørgrav, (Sat Jan 5, 10:16 am)
Re: ELF dynamic loader name, Tim Kientzle, (Sun Jan 13, 3:33 pm)
Re: ELF dynamic loader name, John Baldwin, (Mon Jan 14, 9:51 am)
Re: ELF dynamic loader name, Dag-Erling Smørgrav, (Sun Jan 13, 3:48 pm)
Re: ELF dynamic loader name, John Baldwin, (Mon Jan 14, 9:46 am)
Re: ELF dynamic loader name, Alexander Leidinger, (Mon Jan 14, 7:03 am)
Re: ELF dynamic loader name [was: sbrk(2) broken] , Danny Braniss, (Sat Jan 5, 3:32 am)
Re: ELF dynamic loader name [was: sbrk(2) broken], Peter Wemm, (Sat Jan 5, 6:24 pm)
Re: ELF dynamic loader name [was: sbrk(2) broken] , Danny Braniss, (Sun Jan 6, 3:56 am)
Re: ELF dynamic loader name [was: sbrk(2) broken], Andrew Reilly, (Sun Jan 6, 4:42 pm)
Re: ELF dynamic loader name, Dag-Erling Smørgrav, (Mon Jan 7, 5:42 am)
Re: ELF dynamic loader name, Andrew Reilly, (Mon Jan 7, 7:30 pm)
Re: ELF dynamic loader name, Dag-Erling Smørgrav, (Tue Jan 8, 5:27 am)
Re: ELF dynamic loader name [was: sbrk(2) broken], Maxim Sobolev, (Fri Jan 4, 7:42 pm)
Re: ELF dynamic loader name [was: sbrk(2) broken], Maxim Sobolev, (Fri Jan 4, 7:38 pm)
Re: ELF dynamic loader name, Dag-Erling Smørgrav, (Sat Jan 5, 10:03 am)
Re: ELF dynamic loader name, Ollivier Robert, (Sat Jan 5, 4:56 pm)