Re: [patch] sched: prevent bound kthreads from changing cpus_allowed

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: David Rientjes <rientjes@...>
Cc: Paul Jackson <pj@...>, <mingo@...>, <peterz@...>, <menage@...>, <linux-kernel@...>, Oleg Nesterov <oleg@...>
Date: Monday, June 9, 2008 - 4:59 pm

David Rientjes wrote:

Actually I have another use case here. Above example in particular may be ok 
but it does demonstrate the issue nicely. Which is that in some cases kthreads 
are bound to a CPU but do not have a strict "must run here" requirement and 
could be moved if needed.
For example I need an ability to move workqueue threads. Workqueue threads do 
kthread_bind().
So how about we add something like kthread_bind_strict() which  would set 
PF_THREAD_BOUND ?
We could also simply add flags argument to the kthread_bind() which would be 
better imo but requires more changes. ie It'd look like
	kthread_bind(..., cpu, KTHREAD_BIND_STRICT);

Things like migration threads, stop machine, etc would use the strict version 
and everything else would use non-strict bind.

---
On the related note (this seems like the right crowd :). What do people think 
about kthreads and cpusets in general. We currently have a bit of a disconnect 
in the logic.
1. kthreads can be put into a cpuset at which point their cpus_allowed mask is 
updated properly
2. kthread's cpus_allowed mask is updated properly when cpuset setup changes 
(cpus added, removed, etc).
3. kthreads inherit cpuset from a parent (kthreadd for example) _but_  they 
either do kthread_bind() or set_cpus_allowed() and both of those simply ignore 
inherited cpusets.

Notice how scenario #3 does not fit into the overall picture. The behaviour is 
inconsistent.
How about this:
- Split sched_setaffinity into

	sched_setaffinity()
	{
		task *p = get_task_by_pid();
		return task_setaffinity(p);
	}

	task_setaffinity(task, cpumask, flags)
	{
		if (flags & FORCE) {
			// Used for kthreads that require strict binding.
			// Detach the task from the current cpuset
			// and put it into the root cpuset.
   			// Set PF_THREAD_BOUND.
		}

		// Rest of the original sched_setaffinity logic
	}

- Have kthreads call task_setaffinity() instead of set_cpus_allowed() directly.
That way the behaviour will be consistent across the board.

Comments ?

Max

--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
Re: [patch] sched: prevent bound kthreads from changing cpus..., Max Krasnyanskiy, (Mon Jun 9, 4:59 pm)
workqueue cpu affinity, Max Krasnyansky, (Tue Jun 10, 4:24 pm)
Re: workqueue cpu affinity, Oleg Nesterov, (Wed Jun 11, 12:08 pm)
Re: workqueue cpu affinity, Max Krasnyansky, (Wed Jun 11, 4:44 pm)
Re: workqueue cpu affinity, Max Krasnyansky, (Wed Jun 11, 3:21 pm)
Re: workqueue cpu affinity, Oleg Nesterov, (Thu Jun 12, 12:35 pm)
Re: workqueue cpu affinity, Max Krasnyansky, (Wed Jun 11, 3:21 pm)
Re: workqueue cpu affinity, Peter Zijlstra, (Wed Jun 11, 2:49 am)
Re: workqueue cpu affinity, Max Krasnyansky, (Wed Jun 11, 3:02 pm)
Re: workqueue cpu affinity, Peter Zijlstra, (Thu Jun 12, 2:44 pm)
Re: workqueue cpu affinity, Max Krasnyanskiy, (Thu Jun 12, 3:10 pm)
cpusets and kthreads, inconsistent behaviour, Max Krasnyansky, (Tue Jun 10, 12:30 pm)
Re: cpusets and kthreads, inconsistent behaviour, David Rientjes, (Tue Jun 10, 2:47 pm)
Re: cpusets and kthreads, inconsistent behaviour, Max Krasnyansky, (Tue Jun 10, 4:44 pm)
Re: cpusets and kthreads, inconsistent behaviour, David Rientjes, (Tue Jun 10, 4:54 pm)
Re: cpusets and kthreads, inconsistent behaviour, Max Krasnyansky, (Tue Jun 10, 5:15 pm)