Re: [PATCH v4] sched: automated per session task groups

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Linus Torvalds
Date: Saturday, December 4, 2010 - 11:33 am

On Sat, Dec 4, 2010 at 9:39 AM, Colin Walters <walters@verbum.org> wrote:

"nice" doesn't work. It never has. Nobody ever uses it, and that has
always been true.

As you note, you can find occasional cases of it being used, but they
are either for things that are _so_ unimportant (and know they are)
and annoying cpu hogs that they wouldn't be allowed to live unless
they were niced down maximally (your tracker-miner example), or they
use nice not because they really want to, but because it is an
approximation for what they really do want (ie pulseaudio wants low
latencies, and is set up by the distro, so you'll find it niced up).

But the fundamental issue is that 'nice' is broken. It's very much
broken at a conceptual and technical design angle (absolute priority
levels, no fairness), but it's broken also from a psychological and
practical angle (ie expecting people to manually do extra work is
ridiculous and totally unrealistic).


Why would you want to do that? If you are willing to do group
scheduling, do it on something sane and meaningful, and something that
doesn't need user interaction or decisions. And do it on something
that has more than 20 levels.

You could, for example, decide to do it per session.


Nobody but morons ever "documented" that. Sure, you can find people
saying it, but you won't be finding people actually _doing_ it. Look
around.

Seriously. Nobody _ever_ does "nice make", unless they are seriously
repressed beta-males (eg MIS people who get shouted at when they do
system maintenance unless they hide in dark corners and don't get
discovered). It just doesn't happen.

But more fundamentally, it's still the wrong thing to do. What nice
level should you use?

And btw, it's not just "make". One of the things that originally
caused me to want something like this is that you can enable some
pretty aggressive threading with "git diff". If you use the
"core.preloadindex" setting, git will fire up 20 threads just to do
"lstat()" system calls as quickly as it humanly can. Or "git grep"
will happily use lots of threads and really mess with your system,
except it limits the threads to a smallish number just to not be
asocial.

Do you want to do "nice git" too? Especially as the reason the
threaded lstat was implemented was that over NFS, you actually want
the threads not because you're using lots of CPU, but because you want
to fire up lots of concurrent network traffic - and you actually want
low latency. So you do NOT want to mark these threads as
"unimportant". They're not.

But what you do want is a basic and automatic fairness. When I do "git
grep", I want the full resources of the machine to do the grep for me,
so that I can get the answer in half a second (which is about the
limit at which point I start getting impatient). That's an _important_
job for me. It should get all the resources it can, there is
absolutely no excuse for nicing it down.

But at the same time, if I just happen to have sound or something
going on at the same time, I would definitely like some amount of
fairness. Just because git is smart and can use lots of threads to do
its work quickly, it shouldn't be _unfair_. It should hod the machine
- but only up to a point of some fairness.

That is something that "nice" can never give you. It's not what nice
was designed for, it's not how nice works. And if you ask people to
say "this work isn't important", you shouldn't expect them to actually
do it. If something isn't important, I certainly won't then spend
extra effort on it, for chrissake!

Now, I'm not saying that cgroups are necessarily the answer either.
But using sessions as input to group scheduling is certainly _one_
answer. And it's a hell of a better answer than 'nice' has ever been,
or will ever be.

                             Linus
--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[RFC/RFT PATCH] sched: automated per tty task groups, Mike Galbraith, (Tue Oct 19, 2:16 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Peter Zijlstra, (Tue Oct 19, 2:26 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Peter Zijlstra, (Tue Oct 19, 2:29 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mike Galbraith, (Tue Oct 19, 2:39 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mike Galbraith, (Tue Oct 19, 2:42 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Peter Zijlstra, (Tue Oct 19, 2:43 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mike Galbraith, (Tue Oct 19, 2:46 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mike Galbraith, (Tue Oct 19, 4:29 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mike Galbraith, (Tue Oct 19, 6:12 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Linus Torvalds, (Tue Oct 19, 8:28 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mike Galbraith, (Tue Oct 19, 11:13 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mike Galbraith, (Tue Oct 19, 11:53 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Markus Trippelsdorf, (Wed Oct 20, 6:55 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mike Galbraith, (Wed Oct 20, 7:41 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mike Galbraith, (Thu Oct 21, 12:55 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mike Galbraith, (Thu Oct 21, 1:11 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mike Galbraith, (Thu Oct 21, 1:39 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Markus Trippelsdorf, (Thu Oct 21, 1:48 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mike Galbraith, (Thu Oct 21, 1:52 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Peter Zijlstra, (Thu Oct 21, 3:28 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mathieu Desnoyers, (Thu Oct 21, 3:51 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Peter Zijlstra, (Thu Oct 21, 4:25 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mike Galbraith, (Thu Oct 21, 4:27 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mathieu Desnoyers, (Thu Oct 21, 9:22 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Oleg Nesterov, (Thu Oct 21, 9:29 am)
Re: [RFC/RFT PATCH] sched: automated per tty task groups, Mike Galbraith, (Thu Oct 21, 12:11 pm)
[RFC/RFT PATCH v3] sched: automated per tty task groups, Mike Galbraith, (Tue Oct 26, 12:07 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Mike Galbraith, (Tue Oct 26, 12:29 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Linus Torvalds, (Thu Nov 11, 11:34 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Mike Galbraith, (Thu Nov 11, 12:08 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Markus Trippelsdorf, (Thu Nov 11, 12:15 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Mike Galbraith, (Thu Nov 11, 12:35 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Linus Torvalds, (Thu Nov 11, 12:37 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Markus Trippelsdorf, (Thu Nov 11, 12:38 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Mike Galbraith, (Thu Nov 11, 12:58 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Mike Galbraith, (Sun Nov 14, 10:19 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Markus Trippelsdorf, (Sun Nov 14, 10:49 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Mike Galbraith, (Sun Nov 14, 11:10 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Linus Torvalds, (Sun Nov 14, 12:28 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Markus Trippelsdorf, (Sun Nov 14, 1:27 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Valdis.Kletnieks, (Mon Nov 15, 3:41 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Tue Nov 16, 10:03 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Linus Torvalds, (Tue Nov 16, 10:11 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Mike Galbraith, (Tue Nov 16, 10:42 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Peter Zijlstra, (Tue Nov 16, 11:08 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Tue Nov 16, 11:16 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Peter Zijlstra, (Tue Nov 16, 11:21 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Linus Torvalds, (Tue Nov 16, 11:49 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Peter Zijlstra, (Tue Nov 16, 11:59 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Peter Zijlstra, (Tue Nov 16, 12:13 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Peter Zijlstra, (Tue Nov 16, 12:25 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Linus Torvalds, (Tue Nov 16, 12:35 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Markus Trippelsdorf, (Tue Nov 16, 12:42 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Peter Zijlstra, (Tue Nov 16, 12:43 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Linus Torvalds, (Tue Nov 16, 12:45 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Linus Torvalds, (Tue Nov 16, 12:49 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Mike Galbraith, (Tue Nov 16, 12:57 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Tue Nov 16, 1:03 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Tue Nov 16, 1:05 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Tue Nov 16, 1:28 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Tue Nov 16, 1:31 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Tue Nov 16, 1:33 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Tue Nov 16, 1:36 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Tue Nov 16, 1:50 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Tue Nov 16, 2:08 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Tue Nov 16, 2:14 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Tue Nov 16, 2:17 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Tue Nov 16, 2:19 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Tue Nov 16, 5:21 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Wed Nov 17, 8:01 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Wed Nov 17, 3:34 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Wed Nov 17, 3:45 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Wed Nov 17, 4:49 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Hans-Peter Jansen, (Thu Nov 18, 3:33 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Samuel Thibault, (Thu Nov 18, 4:12 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Samuel Thibault, (Thu Nov 18, 4:43 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Samuel Thibault, (Thu Nov 18, 5:02 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Samuel Thibault, (Thu Nov 18, 5:07 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Samuel Thibault, (Thu Nov 18, 5:42 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Samuel Thibault, (Thu Nov 18, 5:59 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Samuel Thibault, (Thu Nov 18, 6:23 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Mathieu Desnoyers, (Thu Nov 18, 8:15 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Samuel Thibault, (Fri Nov 19, 2:02 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Mathieu Desnoyers, (Fri Nov 19, 5:55 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Mathieu Desnoyers, (Fri Nov 19, 6:20 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Samuel Thibault, (Fri Nov 19, 7:24 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Samuel Thibault, (Fri Nov 19, 7:55 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Fri Nov 19, 9:34 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Linus Torvalds, (Fri Nov 19, 10:51 am)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Mike Galbraith, (Fri Nov 19, 12:31 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Linus Torvalds, (Fri Nov 19, 12:48 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Fri Nov 19, 6:13 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Fri Nov 19, 6:33 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Sat Nov 20, 8:41 am)
[PATCH v4] sched: automated per session task groups, Mike Galbraith, (Sat Nov 20, 12:35 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Mike Galbraith, (Sat Nov 20, 12:47 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Mike Galbraith, (Sat Nov 20, 12:51 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Samuel Thibault, (Sat Nov 20, 1:25 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Konstantin Svist, (Sat Nov 20, 3:02 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Samuel Thibault, (Sat Nov 20, 3:15 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Thomas Fjellstrom, (Sat Nov 20, 3:18 pm)
Re: [RFC/RFT PATCH v3] sched: automated per tty task groups, Lennart Poettering, (Mon Nov 22, 12:21 pm)
[tip:sched/core] sched: Add 'autogroup' scheduling feature ..., tip-bot for Mike Gal ..., (Tue Nov 30, 8:39 am)
Re: [PATCH v4] sched: automated per session task groups, Colin Walters, (Sat Dec 4, 10:39 am)
Re: [PATCH v4] sched: automated per session task groups, Linus Torvalds, (Sat Dec 4, 11:33 am)
Re: [PATCH v4] sched: automated per session task groups, Colin Walters, (Sat Dec 4, 1:01 pm)
Re: [PATCH v4] sched: automated per session task groups, Linus Torvalds, (Sat Dec 4, 3:39 pm)
Re: [PATCH v4] sched: automated per session task groups, Colin Walters, (Sat Dec 4, 4:43 pm)
Re: [PATCH v4] sched: automated per session task groups, Linus Torvalds, (Sat Dec 4, 5:31 pm)
Re: [PATCH v4] sched: automated per session task groups, Nikos Chantziaras, (Sun Dec 5, 4:11 am)
Re: [PATCH v4] sched: automated per session task groups, Mike Galbraith, (Sun Dec 5, 4:36 am)
Re: [PATCH v4] sched: automated per session task groups, Colin Walters, (Sun Dec 5, 12:22 pm)
Re: [PATCH v4] sched: automated per session task groups, Linus Torvalds, (Sun Dec 5, 1:47 pm)
Re: [PATCH v4] sched: automated per session task groups, Colin Walters, (Sun Dec 5, 3:47 pm)
Re: [PATCH v4] sched: automated per session task groups, Valdis.Kletnieks, (Sun Dec 5, 5:28 pm)
Re: [PATCH v4] sched: automated per session task groups, Peter Zijlstra, (Tue Dec 7, 11:51 am)
[tip:sched/core] sched, autogroup: Fix potential access to ..., tip-bot for Mike Gal ..., (Tue Jan 4, 7:18 am)