Re: [RFC 0/4] dynamically allocate arch specific system vectors

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Jack Steiner
Date: Wednesday, September 17, 2008 - 1:21 pm

On Wed, Sep 17, 2008 at 12:15:42PM -0700, H. Peter Anvin wrote:

It is one vector for each cpu.

It is more efficient for software if the vector # is the same for all cpus
but the software/hardware can support a unique vector for each cpu. This
assumes, of course, that the driver can determine the irq->vector mapping for
each cpu.


<probably-more-detail-than-you-want>

Physically, the system contains a large number of blades. Each blade has
several processor sockets plus a UV hub (node controller).  There are 2 GRUs
located in each UV hub.

Each GRU supports multiple users simultaneously using the GRU.
Each user is assigned a context number (0 .. N-1). If an exception occurs,
the GRU uses the context number as an index into an array of [vector-apicid] pairs.
The [vector-apicid] identifies the cpu & vector for the interrupt.

Although supported by hardware, we do not intend to send interrupts
off-blade.

The array of [vector-apicid] pairs is located in each GRU and must be
initialized at boot time or when the driver is loaded. There is a
separate array for each GRU.

When the driver receives the interrupt, the vector number (or IRQ number) is
used by the driver to determine the GRU that sent the interrupt.


The simpliest scheme would be to assign 2 vectors - one for each GRU in the UV hub.
Vector #0 would be loaded into each "vector" of the  [vector-apicid] array for GRU
#0; vector #1 would be loaded into the [vector-apicid] array for GRU #1.

The [vector-apicid] arrays on all nodes would be identical as far as vectors are
concerned. (Apicids would be different and would target blade-local cpus).
Since interrupts are not sent offnode, the driver can use the vector (irq)
to uniquely identify the source of the interrupt.

However, we have a lot of flexibilty here. Any scheme that provides the right
information to the driver is ok. Note that servicing of these interrupts
is likely to be time critical. We need this path to be as efficient as possible.



--- jack




--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
Re: [PATCH] x86_64: (NEW) Dynamically allocate arch speci ..., Eric W. Biederman, (Mon Aug 11, 12:39 pm)
Re: [PATCH] x86_64: (NEW) Dynamically allocate arch speci ..., Jeremy Fitzhardinge, (Mon Aug 11, 12:55 pm)
Re: [PATCH] x86_64: (NEW) Dynamically allocate arch speci ..., Eric W. Biederman, (Mon Aug 11, 1:10 pm)
Re: [RFC 0/4] dynamically allocate arch specific system ve ..., Dimitri Sivanich, (Wed Sep 17, 10:30 am)
Re: [RFC 0/4] dynamically allocate arch specific system ve ..., Eric W. Biederman, (Wed Sep 17, 11:59 am)
Re: [RFC 0/4] dynamically allocate arch specific system ve ..., Jack Steiner, (Wed Sep 17, 1:21 pm)
Re: [RFC 0/4] dynamically allocate arch specific system ve ..., Eric W. Biederman, (Wed Sep 17, 3:15 pm)
Re: [RFC 0/4] dynamically allocate arch specific system ve ..., Eric W. Biederman, (Thu Sep 18, 5:28 pm)