[PATCH] kernel/cpu.c: Move the CPU_DYING notifiers

Previous thread: [BUG] x86 kenel won't boot under Virtual PC by David Sanders on Sunday, August 31, 2008 - 11:22 am. (21 messages)

Next thread: Re: [PATCH] x86: split e820 reserved entries record to late v2 by David Witbrodt on Sunday, August 31, 2008 - 12:10 pm. (1 message)
From: Manfred Spraul
Date: Sunday, August 31, 2008 - 10:58 am

When a cpu is taken offline, the CPU_DYING notifiers are called on the
dying cpu. According to <linux/notifiers.h>, the cpu should be "not
running any task, not handling interrupts, soon dead".

For the current implementation, this is not true:
- __cpu_disable can fail. If it fails, then the cpu will remain alive
  and happy.
- At least on x86, __cpu_disable() briefly enables the local interrupts
  to handle any outstanding interrupts.

What about moving CPU_DYING down a few lines, behind the __cpu_disable()
line?
There are only two CPU_DYING handlers in the kernel right now: one in
kvm, one in the scheduler. Both should work with the patch applied
[and: I'm not sure if either one handles a failing __cpu_disable()]

The patch survives simple offlining a cpu. kvm untested due to lack
of a test setup.

Signed-Off-By: Manfred Spraul <manfred@colorfullife.com>
---
 kernel/cpu.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index e202a68..5b7c88f 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -199,13 +199,14 @@ static int __ref take_cpu_down(void *_param)
 	struct take_cpu_down_param *param = _param;
 	int err;
 
-	raw_notifier_call_chain(&cpu_chain, CPU_DYING | param->mod,
-				param->hcpu);
 	/* Ensure this CPU doesn't handle any more interrupts. */
 	err = __cpu_disable();
 	if (err < 0)
 		return err;
 
+	raw_notifier_call_chain(&cpu_chain, CPU_DYING | param->mod,
+				param->hcpu);
+
 	/* Force idle task to run as soon as we yield: it should
 	   immediately notice cpu is offline and die quickly. */
 	sched_idle_next();
-- 
1.5.5.1

--

From: Paul E. McKenney
Date: Sunday, August 31, 2008 - 12:17 pm

Several architectures re-enable interrupts in __cpu_disable() or in
functions called from __cpu_disable(), which happens after CPU_DYING,
if I understand correctly.  :-(

--

From: Paul E. McKenney
Date: Sunday, August 31, 2008 - 12:23 pm

Never mind -- you are moving CPU_DYING after __cpu_disable().  :-/

--

From: Ingo Molnar
Date: Saturday, September 6, 2008 - 9:49 am

hm, doesnt this break things like CPU cross-calls done in CPU_DYING 
callbacks?

	Ingo
--

From: Manfred Spraul
Date: Saturday, September 6, 2008 - 10:08 am

We are within stop_machine(). No other cpu is running. As fas as I can 
see no cross-calls are possible.

Which scenario do you think about?

--
    Manfred
--

From: Ingo Molnar
Date: Saturday, September 6, 2008 - 10:13 am

ah, ok - my bad. I was confusing it with the much more common 
CPU_DOWN_PREPARE type of callbacks which do use various cross-CPU APIs.

applied to tip/sched/devel, thanks Manfred!

	Ingo
--

From: Avi Kivity
Date: Friday, September 12, 2008 - 11:36 pm

kvm should work with this patch.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

--

Previous thread: [BUG] x86 kenel won't boot under Virtual PC by David Sanders on Sunday, August 31, 2008 - 11:22 am. (21 messages)

Next thread: Re: [PATCH] x86: split e820 reserved entries record to late v2 by David Witbrodt on Sunday, August 31, 2008 - 12:10 pm. (1 message)