Re: [PATCH 2/2] x86_64 irq: Handle irqs pending in IRR during irq migration.

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Eric W. Biederman
Date: Thursday, February 8, 2007 - 11:40 pm

>

Bah.  I had not taken into account that the local apic despite
being tightly coupled with the cpu is for programming purposes
an asynchronous device.  If I want to give it time to react to something
I need to read from it.

The routine below actually works. 

My remaining practical question is can this been done cleanly.

Ingo's lock debugging dislikes this routine.  
By using raw_local_irq_enable I have avoided all but a message on
the irq return path, I haven't quite worked out where.

But at least this version feels like it could be done better
(less inline? different helpers?) someone.

For interrupts coming through a sane interrupt controller moving
this into process context would certainly simplify things.  For edge
triggered interrupts coming through an io_apic I'm not at all certain
what makes sense.

When the routine below is used to ack an edge triggered interrupt
it runs before the edge triggered interrupt handler so losing an
edge shouldn't happen (we haven't acknowledged the hardware yet)
and even if we do the device driver gets to run at least once.

So doing migration in the irq handler still looks like the best
solution even if it is ugly.  As long as the little bit of
stack overhead isn't a problem I think enabling interrupts to
clear out any pending irqs certainly looks simpler. 

In another vein.  I went and looked through all of Intel's and
AMD's public errata that I could find and there weren't any associated
with irr or isr, so I think my previous version of the code is still
sane, and not likely to break.

I can improve it a little by getting the vector as:
"vector = ~ get_irq_regs()->orig_rax;" instead of reading
ISR.  That still leaves reading the pending bit in ISR and the
other funny tricks.

I'm conflicted between the two approaches a little because playing
games with enabling interrupts in an interrupt handler seems to
have some weird corner cases.

static void ack_apic(unsigned int irq)
{
#if defined(CONFIG_GENERIC_PENDING_IRQ) || defined(CONFIG_IRQBALANCE)
	struct irq_desc *desc;
	desc = irq_desc + irq;
	if (likely(!(desc->status & IRQ_MOVE_PENDING)))
		goto simple;

	if (hardirq_count() != HARDIRQ_OFFSET)
		goto simple;

	desc->chip->mask(irq);
	ack_APIC_irq();

	/* Ensure all of the irq handlers for this irq have completed
	 * before we migrate it.
	 */
	spin_unlock(&desc->lock);
	raw_local_irq_enable();
	apic_read(APIC_ID);
	raw_local_irq_disable();
	spin_lock(&desc->lock);

	move_masked_irq(irq);
	desc->chip->unmask(irq);
	return;
simple:
#endif
	ack_APIC_irq();
}


BUG: at /home/eric/projects/linux/linux-2.6-devel/kernel/lockdep.c:1860 trace_hardirqs_on()

Call Trace:
 <IRQ>  [<ffffffff8048562f>] trace_hardirqs_on_thunk+0x35/0x37
 [<ffffffff80290401>] generic_delete_inode+0x0/0x13e
 [<ffffffff8020a0fc>] restore_args+0x0/0x30
 [<ffffffff80290401>] generic_delete_inode+0x0/0x13e
 [<ffffffff8021648d>] ack_apic+0x63/0x99
 [<ffffffff80216485>] ack_apic+0x5b/0x99
 [<ffffffff8025881e>] handle_fasteoi_irq+0xc1/0xd1
 [<ffffffff80290401>] generic_delete_inode+0x0/0x13e
 [<ffffffff8020c0de>] do_IRQ+0x89/0xf3
 [<ffffffff80208ce8>] default_idle+0x35/0x51
 [<ffffffff80208cb3>] default_idle+0x0/0x51
 [<ffffffff8020a0a6>] ret_from_intr+0x0/0xf
 <EOI>  [<ffffffff80290401>] generic_delete_inode+0x0/0x13e
 [<ffffffff80208cb3>] default_idle+0x0/0x51
 [<ffffffff80208ce8>] default_idle+0x35/0x51
 [<ffffffff80208cea>] default_idle+0x37/0x51
 [<ffffffff80208ce8>] default_idle+0x35/0x51
 [<ffffffff80208d5a>] cpu_idle+0x56/0x75
 [<ffffffff808b9a69>] start_secondary+0x481/0x490

-
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[PATCH 1/2] x86_64 irq: Simplfy __assign_irq_vector, Eric W. Biederman, (Fri Feb 2, 5:31 pm)
[PATCH 2/2] x86_64 irq: Handle irqs pending in IRR during ..., Eric W. Biederman, (Fri Feb 2, 5:35 pm)
Re: [PATCH 2/2] x86_64 irq: Handle irqs pending in IRR du ..., Eric W. Biederman, (Fri Feb 2, 6:39 pm)
Re: [PATCH 2/2] x86_64 irq: Handle irqs pending in IRR du ..., Arjan van de Ven, (Sat Feb 3, 12:32 am)
Re: [PATCH 2/2] x86_64 irq: Handle irqs pending in IRR du ..., Eric W. Biederman, (Sat Feb 3, 12:55 am)
Re: [PATCH 2/2] x86_64 irq: Handle irqs pending in IRR du ..., Eric W. Biederman, (Sat Feb 3, 3:22 am)
Re: [PATCH 2/2] x86_64 irq: Handle irqs pending in IRR du ..., Eric W. Biederman, (Tue Feb 6, 1:57 am)
Re: [PATCH 2/2] x86_64 irq: Handle irqs pending in IRR du ..., Eric W. Biederman, (Tue Feb 6, 3:16 pm)
Re: [PATCH 2/2] x86_64 irq: Handle irqs pending in IRR du ..., Eric W. Biederman, (Tue Feb 6, 7:33 pm)
Re: [PATCH 2/2] x86_64 irq: Handle irqs pending in IRR du ..., Eric W. Biederman, (Thu Feb 8, 4:48 am)
Re: [PATCH 2/2] x86_64 irq: Handle irqs pending in IRR du ..., Eric W. Biederman, (Thu Feb 8, 1:19 pm)
Re: [PATCH 2/2] x86_64 irq: Handle irqs pending in IRR du ..., Eric W. Biederman, (Thu Feb 8, 11:40 pm)
What are the real ioapic rte programming constraints?, Eric W. Biederman, (Sat Feb 10, 4:52 pm)
Re: What are the real ioapic rte programming constraints?, Zwane Mwaikambo, (Sat Feb 10, 10:57 pm)
Re: What are the real ioapic rte programming constraints?, Eric W. Biederman, (Sun Feb 11, 3:20 am)
Re: What are the real ioapic rte programming constraints?, Zwane Mwaikambo, (Sun Feb 11, 9:16 am)
Re: What are the real ioapic rte programming constraints?, Eric W. Biederman, (Sun Feb 11, 3:01 pm)
Re: What are the real ioapic rte programming constraints?, Zwane Mwaikambo, (Sun Feb 11, 6:05 pm)
Re: What are the real ioapic rte programming constraints?, Eric W. Biederman, (Sun Feb 11, 9:51 pm)
Conclusions from my investigation about ioapic programming, Eric W. Biederman, (Fri Feb 23, 3:51 am)
[PATCH 0/14] x86_64 irq related fixes and cleanups., Eric W. Biederman, (Fri Feb 23, 4:10 am)
[PATCH 01/14] x86_64 irq: Simplfy __assign_irq_vector, Eric W. Biederman, (Fri Feb 23, 4:11 am)
[PATCH 02/14] irq: Remove set_native_irq_info, Eric W. Biederman, (Fri Feb 23, 4:13 am)
[PATCH 03/14] x86_64 irq: Kill declaration of removed arra ..., Eric W. Biederman, (Fri Feb 23, 4:15 am)
[PATCH 04/14] x86_64 irq: Remove the unused vector paramet ..., Eric W. Biederman, (Fri Feb 23, 4:16 am)
[PATCH 05/14] x86_64 irq: Refactor setup_IO_APIC_irq, Eric W. Biederman, (Fri Feb 23, 4:19 am)
[PATCH 06/14] x86_64 irq: Simplfiy the set_affinity logic., Eric W. Biederman, (Fri Feb 23, 4:20 am)
[PATCH 07/14] x86_64 irq: In __DO_ACTION perform the FINAL ..., Eric W. Biederman, (Fri Feb 23, 4:23 am)
[PATCH 08/14] x86_64 irq: Use NR_IRQS not NR_IRQ_VECTORS, Eric W. Biederman, (Fri Feb 23, 4:26 am)
[PATCH 09/14] x86_64 irq: Begin consolidating per_irq data ..., Eric W. Biederman, (Fri Feb 23, 4:32 am)
[PATCH 10/14] x86_64 irq: Simplify assign_irq_vector's arg ..., Eric W. Biederman, (Fri Feb 23, 4:35 am)
[PATCH 11/14] x86_64 irq: Remove unnecessary irq 0 setup., Eric W. Biederman, (Fri Feb 23, 4:36 am)
[PATCH 12/14] x86_64 irq: Add constants for the reserved I ..., Eric W. Biederman, (Fri Feb 23, 4:38 am)
[PATCH 13/14] x86_64 irq: Safely cleanup an irq after movi ..., Eric W. Biederman, (Fri Feb 23, 4:40 am)
[PATCH 14/14] genirq: Mask irqs when migrating them., Eric W. Biederman, (Fri Feb 23, 4:46 am)
[PATCH] x86_64 irq: Document what works and why on ioapics., Eric W. Biederman, (Fri Feb 23, 5:01 am)
Re: Conclusions from my investigation about ioapic programming, Eric W. Biederman, (Fri Feb 23, 11:10 am)
Re: [PATCH 14/14] genirq: Mask irqs when migrating them., Siddha, Suresh B, (Fri Feb 23, 7:06 pm)
Re: Conclusions from my investigation about ioapic programming, Eric W. Biederman, (Fri Feb 23, 9:05 pm)
Re: Conclusions from my investigation about ioapic programming, Jeffrey V. Merkey, (Fri Feb 23, 10:44 pm)
Re: [PATCH 12/14] x86_64 irq: Add constants for the reserv ..., Eric W. Biederman, (Sun Feb 25, 4:15 am)
Re: [PATCH 13/14] x86_64 irq: Safely cleanup an irq after ..., Eric W. Biederman, (Sun Feb 25, 5:09 am)
Re: [PATCH 12/14] x86_64 irq: Add constants for the reserv ..., Eric W. Biederman, (Sun Feb 25, 2:01 pm)
Re: [PATCH 14/14] genirq: Mask irqs when migrating them., Andrew Morton, (Tue Feb 27, 1:26 pm)
Re: [PATCH 14/14] genirq: Mask irqs when migrating them., Eric W. Biederman, (Tue Feb 27, 1:41 pm)