Re: OpenBSD 4.2 Question

Previous thread: Pretty hard times with openbsd yp server by John Nietzsche on Thursday, September 6, 2007 - 7:07 pm. (1 message)

Next thread: Re: Show your appreciation and get your 4.2 DVD by Greg Thomas on Thursday, September 6, 2007 - 10:58 pm. (9 messages)
To: misc <misc@...>
Date: Thursday, September 6, 2007 - 10:31 pm

hello misc@
from the page http://www.openbsd.org/42.html , one of the changes made
to OpenBSD 4.2 is

A change in the way the kernel random pool is stirred greatly
increases performance with network interface cards that support
interrupt mitigation, especially on architectures where reading the
clock is expensive (such as amd64).

What would be some Examples of Network Cards that Support "interrupt mitigation"

I guess on this Subject I need educated because I am not all together
sure what interrupt mitigation is and why I want it.

Thank you for another GREAT release

Sam Fourman Jr.

To: Sam Fourman Jr. <sfourman@...>
Cc: misc <misc@...>
Date: Friday, September 7, 2007 - 8:26 am

I do want to point out that interrupt coalescing also has its draw
backs. Many people discussed all the good things so lets underscore the
bad. Networks that have trickling amounts of packets are hurt by
coalescing. Most notably are interactive things like typing in ftp and
ssh. Since the card is going to collect several packets you basically
always hit the coalescing timer slowing the overall throughput down to
whatever your timeout is.

The big issue with coalescing is that it is good for some loads and bad
for others. So if you can't predict your load it will either hurt you
or help you. So far I have never seen a heuristic that works well both
ways.

Also what I have seen is that coalescing always hurts disk IO
performance. Every time I tinkered with this it had a negative impact.
The caveat here is that NICs generate interrupts at a completely
different order of magnitude.

To: Marco Peereboom <slash@...>
Cc: misc <misc@...>
Date: Saturday, September 8, 2007 - 12:48 am

Thank you Everyone for your response, the information was very helpful

Sam Fourman Jr.

To: misc <misc@...>
Date: Friday, September 7, 2007 - 6:40 am

traditionally, and interrupt cards intterupts once per packet it has
received (or successfully sent, but let us ignore the tx path for now).
interrupt mitigation, coalescing, *insert name-of-the-day for int
mitgation here* etc all describe teh same thing, a certain heuristic to
not interrupt once per packet, but once for a bunch. this is usually
achieved by a combination of amount of packets (i. e. "interrupt every
25 packets") or bytes and a timer (to make sure there is no packet
sitting in the rx queue for too long).

--
Henning Brauer, hb@bsws.de, henning@openbsd.org
BS Web Services, http://bsws.de
Full-Service ISP - Secure Hosting, Mail and DNS Services
Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam

To: misc <misc@...>
Date: Friday, September 7, 2007 - 7:48 am

let me try to translate my mail to english...

and let me explain why that makes so much of a difference. rocessing
the packets is one thing, context switching between the interrupt
handler (which takes teh packet from the nic's rx ring and puts it into
the ipintrq), the soft interrupt handling (where the packet is taken
from the ipintrq and running through ip_input, then maybe ip_forward and
_output and put into the (potentially other) nic's output queue, well,
that switching is not exactly free, performance wise, to put it nicely.
so instead of interrupt, one packet into ipintrq, processing at softnet,
oh wait interrupt, peel another out of the rx ring, etc yadda yadda it
is much cheaper to process a bunch of packets at once.

the downside is that you potentially increase latency, since after
packet reception teh nic does not interupt immediately and you don't
process immediately (for some value of immediately), but only after
enough othe rpackets are there. to cope with that effect the time I
mentioned above is there, so that a apcket doesn't sit in teh rx ring
for longer than XXX msec, wether otehrsd arrive or not.

--
Henning Brauer, hb@bsws.de, henning@openbsd.org
BS Web Services, http://bsws.de
Full-Service ISP - Secure Hosting, Mail and DNS Services
Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam

To: <misc@...>
Date: Friday, September 7, 2007 - 1:24 am

Hello everybody,

I need install two QLA2462-SUN-XO HBA cards in a AMD SunFire V40Z
server, but I can find information about this card support by the isp
driver, anyone know about this card support in OpenBSD 4.1

Thanks and regards,

__________________
Julian Bolivar
www.julianbolivar.com

To: Sam Fourman Jr. <sfourman@...>
Cc: misc <misc@...>
Date: Friday, September 7, 2007 - 12:26 am

lots of them, though they might call it different things. sometimes
the hardware may "support" a "feature" but we don't enable it because
it hurts more than it helps.

by way of example, fxp, em, bge, and sis (to name a few) support such
a thing. grepping for
strings like {delay, hold, coales, interrupt, throttl, mitigat} in the

let's say your NIC fires an interrupt every time you get/send a packet
- the OS has to stop whatever it was doing, figure out who sent the
interrupt, decide what to do about it, and then go do it. Kind of like
a telephone call.

At low packet rates this is not a problem ... millions of processor
cycles between interrupts means plenty of time to get work done. Now
we crank that up a few orders of magnitude... suddenly your CPU is so
busy responding to these interrupts that it never has time to get any
real work done. Kind of like a telephone "ringing off the hook"

Enter interrupt mitigation. In general this allows the card to delay
the interrupt for a short while, just in case more packets arrive
right away. An interrupt happens, but this time there are ... say,
8... packets to handle. The overhead of handling the interrupt has
been effectively reduced to 1/8, at the cost of introducing a small
amount of delay. Kind of like sending your calls to voicemail and
returning calls in a batch once an hour.

Too little delay, and your kernel can get run into the ground by
interrupts. Too much delay and your network sucks because everybody is
waiting for your network card to tell the kernel "hey, i have packets
for you", or you run out of buffers on the card and start dropping
packets.

tuning interrupt mitigation is a bit tricky. one example of interrupt
migitation gone badly might be Vista's apparent inability to
simultaneously play sound and do high speed net I/O. the idea was ok -
prevent net I/O from totally killing more time-sensitive processes.
the execution was flawed...

--
GDB has a 'break' feature; why doesn't it have 'fix' too?

To: Sam Fourman Jr. <sfourman@...>
Cc: misc <misc@...>
Date: Thursday, September 6, 2007 - 11:20 pm

I'm not the right person to try to explain it, but I have generally
been interested in seeing where it crops up and what it's associated
with.

The various /plusXY.html pages often have notes indicating drivers for
which interrupt mitigation has been enabled/enhanced, and you can see
what generally has it. For example, from plus38.html:

"Add microcode to support interrupt mitigation on fxp(4) 82551 F
stepping chipset. Big performance boost."

plus39.html:
"Fix for the nfe(4) interrupt mitigation code."

...or a CVS commit about enabling it in a National Semiconductor chip:
http://marc.info/?l=openbsd-cvs&m=118037514315592&w=2

...after which Soekris boxen with the DP83816 chips could experience
better throughput at lower interrupt rates, good since the poor
thing's CPU is a limiting factor. It's fun to look for this kind of
stuff in the CVS commit logs, e.g.

http://marc.info/?l=openbsd-cvs&w=2&r=1&s=%22interrupt+mitigation%22&q=b
http://marc.info/?l=openbsd-cvs&w=2&r=1&s=%22interrupt+holdoff%22&q=b
http://marc.info/?l=openbsd-cvs&w=2&r=1&s=interrupt+coalesce&q=b
and other places:
http://www.openbsd.org/papers/cuug2007/mgp00016.txt

DS

Previous thread: Pretty hard times with openbsd yp server by John Nietzsche on Thursday, September 6, 2007 - 7:07 pm. (1 message)

Next thread: Re: Show your appreciation and get your 4.2 DVD by Greg Thomas on Thursday, September 6, 2007 - 10:58 pm. (9 messages)