Re: [patch] sched: prevent bound kthreads from changing cpus_allowed

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: David Rientjes <rientjes@...>
Cc: Paul Jackson <pj@...>, <mingo@...>, <peterz@...>, <menage@...>, <linux-kernel@...>, Oleg Nesterov <oleg@...>
Date: Tuesday, June 10, 2008 - 12:23 am

David Rientjes wrote:
Yes.

I think what can and cannot be moved is a separate question. As far as cpu
affinity API for kthreads goes it makes sense to support both scenarios.
See below.

CPU isolation. ie To avoid kernel activity on certain CPUs.

That would be inconsistent and confusing, I think. If a task is in the cpuset
X all constraints of the cpuset X must apply. If some constraints cannot be
satisfied then that task should not be in the cpuset X.

Yes cpusets are not only about cpu affinity. But again the behaviour should be
consistent across the board. cpuset.cpus must apply to all the task in the
set, not just some of the tasks.

Also I think you missed my other point/suggestion. Yes your patch fixes one
problem, which is kthreads that must not move will not move. Although like I
said the behaviour with regards to the cpuset is not consistent and confusing.
The other thing that I pointed out is that kthreads that _can_ move do not
honor cpuset constrains.
Let me give another example. Forget the workqueues for a second. Kthreads like
scsi_eh, kswapd, kacpid, etc, etc are all started with cpus_allows=ALL_CPUS
even though they inherit a cpuset from kthreadd. Yes I can move them manually
but the behaviour is not consistent, and for no good reason. kthreads can be
stopped/started at any time (module load for example) which means that I'd
have to keep moving them.

To sum it up here is what I'm suggesting:
kthread_bind(task, cpu)
{
   // Set PF_THREAD_BOUND
   // Move into root cpuset
   // Bind to the cpu
}

kthread_setaffinity(task, cpumask)
{
   // Enforce cpuset.cpus_allowed
   // Updated affinity mask and migrate kthread (if needed)
}

Instead of calling set_cpus_allowed_ptr() kthreads that do not need strict cpu
binding will be calling kthread_setaffinity().
That way the behaviour is consistent across the board.

Makes sense ?

Max
--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
workqueue cpu affinity, Max Krasnyansky, (Tue Jun 10, 4:24 pm)
Re: workqueue cpu affinity, Oleg Nesterov, (Wed Jun 11, 12:08 pm)
Re: workqueue cpu affinity, Max Krasnyansky, (Wed Jun 11, 4:44 pm)
Re: workqueue cpu affinity, Max Krasnyansky, (Wed Jun 11, 3:21 pm)
Re: workqueue cpu affinity, Oleg Nesterov, (Thu Jun 12, 12:35 pm)
Re: workqueue cpu affinity, Max Krasnyansky, (Wed Jun 11, 3:21 pm)
Re: workqueue cpu affinity, Peter Zijlstra, (Wed Jun 11, 2:49 am)
Re: workqueue cpu affinity, Max Krasnyansky, (Wed Jun 11, 3:02 pm)
Re: workqueue cpu affinity, Peter Zijlstra, (Thu Jun 12, 2:44 pm)
Re: workqueue cpu affinity, Max Krasnyanskiy, (Thu Jun 12, 3:10 pm)
cpusets and kthreads, inconsistent behaviour, Max Krasnyansky, (Tue Jun 10, 12:30 pm)
Re: cpusets and kthreads, inconsistent behaviour, David Rientjes, (Tue Jun 10, 2:47 pm)
Re: cpusets and kthreads, inconsistent behaviour, Max Krasnyansky, (Tue Jun 10, 4:44 pm)
Re: cpusets and kthreads, inconsistent behaviour, David Rientjes, (Tue Jun 10, 4:54 pm)
Re: cpusets and kthreads, inconsistent behaviour, Max Krasnyansky, (Tue Jun 10, 5:15 pm)
Re: [patch] sched: prevent bound kthreads from changing cpus..., Max Krasnyansky, (Tue Jun 10, 12:23 am)