Re: [RFC/PATCH] cpuset: cpuset irq affinities

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Peter Zijlstra
Date: Thursday, March 6, 2008 - 8:21 am

On Thu, 2008-03-06 at 07:47 -0600, Paul Jackson wrote:

Likely; if for instance you'd want some unbound kernel threads to join
in that overlapping set, then perhaps that name would be badly chosen.

Although I'm not sure which unbound kernel threads would benefit from
such treatment.


Perhaps we're talking about something else here; how bad would it be to
require:

for irq in `cat /cgroup/boot/irqs` ; do echo $irq > /cgroup/irqs; done

be added to rc.local or fully replace your home-brew boot cpuset script?
Its basically an update for that script, giving the exact same semantics
to its user, but moving the larger part of it in-kernel.


Sure, but your application vendors will need to re-certify their
applications to run on new distros, sometimes even re-compile because
ABI changes and the like. Certainly providing a new script in the new
version certified for a new distro isn't too much work?


Assign a map of cpus where irqs will default into, and a way to
explicitly move them out of it.






So, yes, cgroups are perhaps awkward because they group tasks whereas
the current problem is grouping IRQs.

Because we're mapping them onto CPUs, cpusets came to mind.

The thing we 'need', is to provide named groups of irqs and for each
such a group specify a cpu mask that is appropriate.

Grouping them makes sense in that we want to make a functional division.
Some IRQs serve a system as a whole, others serve a subset. Typical
subsets could be a RT process space bounded to a cpu/mem domain.

Other usable subsets could be limiting the IRQs of node local network
and IO cards to the cpu/mem domain that runs the application that uses
them.

So we group irqs like:

  system_on_nodes_1_2_and_3 (default)
  big_io_app_on_nodes_2_and_3
  rt_app_on_node_4

Where, again, you see a strong similarity to the cpu/mem divisions
already made by cpusets.

I understand its somewhat at odds with the hierarchical nature of the
<task, cpu/mem> mapping you currently have. But its not too far away
either.

Do you see what we want to do, and why our - perhaps misguided - choice
for cpusets? Creating a whole new cgroup controller is also weird
because we don't deal in tasks.

[ Aside from grouping existing IRQs we also want to provide a way to
designate a default group for new IRQs. But I think such functionality
will fall into place once we can agree on the rest.]

--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[RFC/PATCH 0/4] CPUSET driven CPU isolation, Peter Zijlstra, (Wed Feb 27, 3:21 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Max Krasnyanskiy, (Wed Feb 27, 4:38 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Ingo Molnar, (Thu Feb 28, 12:50 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Paul Jackson, (Thu Feb 28, 1:08 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Ingo Molnar, (Thu Feb 28, 2:08 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Paul Jackson, (Thu Feb 28, 2:17 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, David Rientjes, (Thu Feb 28, 2:32 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, David Rientjes, (Thu Feb 28, 3:12 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Peter Zijlstra, (Thu Feb 28, 3:19 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Peter Zijlstra, (Thu Feb 28, 3:26 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Ingo Molnar, (Thu Feb 28, 3:46 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Mark Hounschell, (Thu Feb 28, 5:12 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Max Krasnyanskiy, (Thu Feb 28, 10:33 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Paul Jackson, (Thu Feb 28, 10:37 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Paul Jackson, (Thu Feb 28, 10:47 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Max Krasnyanskiy, (Thu Feb 28, 10:48 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Max Krasnyansky, (Thu Feb 28, 12:57 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Max Krasnyansky, (Thu Feb 28, 1:11 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Paul Jackson, (Thu Feb 28, 1:13 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Max Krasnyansky, (Thu Feb 28, 1:23 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Max Krasnyansky, (Thu Feb 28, 1:26 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Paul Jackson, (Thu Feb 28, 1:27 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Max Krasnyansky, (Thu Feb 28, 1:45 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, David Rientjes, (Thu Feb 28, 2:24 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Paul Jackson, (Thu Feb 28, 3:46 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, David Rientjes, (Thu Feb 28, 4:00 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Paul Jackson, (Thu Feb 28, 5:16 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, David Rientjes, (Thu Feb 28, 6:05 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Paul Jackson, (Thu Feb 28, 8:34 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, David Rientjes, (Thu Feb 28, 9:00 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Paul Jackson, (Thu Feb 28, 11:53 pm)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Andrew Morton, (Fri Feb 29, 1:31 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Andrew Morton, (Fri Feb 29, 1:36 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Ingo Molnar, (Fri Feb 29, 2:10 am)
Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation, Max Krasnyanskiy, (Fri Feb 29, 11:06 am)
[RFC/PATCH] cpuset: cpuset irq affinities, Peter Zijlstra, (Fri Feb 29, 11:55 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Ingo Molnar, (Fri Feb 29, 12:02 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Max Krasnyanskiy, (Fri Feb 29, 1:52 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Fri Feb 29, 1:55 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Peter Zijlstra, (Fri Feb 29, 2:03 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Peter Zijlstra, (Fri Feb 29, 2:14 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Max Krasnyanskiy, (Fri Feb 29, 2:20 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Ingo Molnar, (Fri Feb 29, 2:29 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Ingo Molnar, (Fri Feb 29, 2:32 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Max Krasnyanskiy, (Fri Feb 29, 2:42 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Fri Feb 29, 2:53 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Fri Feb 29, 3:00 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Christoph Hellwig, (Sat Mar 1, 10:18 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Peter Zijlstra, (Mon Mar 3, 4:57 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Mon Mar 3, 10:36 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Peter Zijlstra, (Mon Mar 3, 10:57 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Mon Mar 3, 11:10 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Peter Zijlstra, (Mon Mar 3, 11:18 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Menage, (Mon Mar 3, 11:41 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Mon Mar 3, 11:52 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Menage, (Mon Mar 3, 10:26 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Mon Mar 3, 11:15 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Menage, (Mon Mar 3, 11:21 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Mon Mar 3, 11:26 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Menage, (Mon Mar 3, 11:34 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Mon Mar 3, 11:51 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Tue Mar 4, 12:35 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Peter Zijlstra, (Tue Mar 4, 4:06 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Max Krasnyanskiy, (Tue Mar 4, 12:52 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Tue Mar 4, 6:11 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Peter Zijlstra, (Wed Mar 5, 1:37 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Ingo Molnar, (Wed Mar 5, 1:50 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Wed Mar 5, 5:35 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Ingo Molnar, (Wed Mar 5, 5:43 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Wed Mar 5, 10:44 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Max Krasnyansky, (Wed Mar 5, 12:17 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Thu Mar 6, 6:47 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Peter Zijlstra, (Thu Mar 6, 8:21 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Thu Mar 6, 8:40 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Thu Mar 6, 11:39 pm)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Menage, (Fri Mar 7, 1:47 am)
Re: [RFC/PATCH] cpuset: cpuset irq affinities, Paul Jackson, (Fri Mar 7, 7:57 am)