Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapter (perf + regression IRQs)

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Jesper Dangaard Brouer
Date: Wednesday, November 19, 2008 - 3:58 pm

On Thu, 13 Nov 2008, David Miller wrote:


[Regression]

Well that was not the real cause of the performance loss.  Because on 
kernel 2.6.27 I get really good performance (900-1200kpps) compared to 
2.6.28 (git net-2.6).

The cause of this problem (tracked down together with Robert Olsson) is 
that on 2.6.28 I have a lot less IRQs available.  It seems max 34 IRQs.

Due the reduced number of IRQs the NIU driver cannot get enough IRQs to 
the interfaces, and starts to use "IO-APIC" based IRQs.

On kernel 2.6.28:

  My eth2 is using 10 IRQs all "PCI-MSI-edge".

  BUT my eth3 is using a single IRQ using "IO-APIC-fasteoi" and shared
  with the usb driver...

Think thats must be my performance problem on 2.6.28.



[Packet performance]

Yes, I know, I do use pktgen and multi-flows (rand dest IP+port).

For the two drivers NIU and Suns NXGE, my packet per sec performance is 
now, on 2.6.27 (with backported NIU fixes).

   With NIU driver I can route 900 kpps.

   With NXGE driver (and enqueue=NULL hack) I can route 1200 kpps.

Actually I think I can go higher, because I'm limited by my packet rate 
generator. I use pktgen (with rand dst IP+port) and can only generate 1200 
kpps.

(I have actually ordered some new hardware, so I can get a faster pktgen 
machine and perhaps test it as a router too.  Also ordered the hardware 
because I want to test PCI-express v.2.0. I have a prototype 12-port 
gigabit NIC (from hotlava systems) that support PCIe v.2.0 and has 6x 
82575 chips (4RX/4TX queues))


Hilsen
   Jesper Brouer

--
-------------------------------------------------------------------
MSc. Master of Computer Science
Dept. of Computer Science, University of Copenhagen
Author of http://www.adsl-optimizer.dk
-------------------------------------------------------------------
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapter, Jesper Dangaard Brouer, (Tue Nov 4, 7:45 am)
Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapter, Jesper Dangaard Brouer, (Wed Nov 5, 12:05 am)
Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapter, Jesper Dangaard Brouer, (Wed Nov 5, 2:30 am)
Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapter, Jesper Dangaard Brouer, (Wed Nov 12, 2:36 am)
Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapter, Jesper Dangaard Brouer, (Wed Nov 12, 3:04 am)
Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapter, Jesper Dangaard Brouer, (Wed Nov 12, 4:01 am)
Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapter, Jesper Dangaard Brouer, (Wed Nov 12, 5:49 am)
Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapter, Jesper Dangaard Brouer, (Wed Nov 12, 6:21 am)
Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapter, Jesper Dangaard Brouer, (Wed Nov 12, 2:31 pm)
Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapter, Jesper Dangaard Brouer, (Thu Nov 13, 1:50 am)
Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapter, Jesper Dangaard Brouer, (Thu Nov 13, 2:10 am)
Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapter, Jesper Dangaard Brouer, (Thu Nov 13, 3:29 am)
Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapt ..., Jesper Dangaard Brouer, (Fri Nov 14, 5:38 am)
Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapt ..., Jesper Dangaard Brouer, (Fri Nov 14, 11:49 am)
Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapt ..., Jesper Dangaard Brouer, (Wed Nov 19, 5:10 am)
Re: NIU driver: Sun x8 Express Quad Gigabit Ethernet Adapt ..., Jesper Dangaard Brouer, (Wed Nov 19, 3:58 pm)
Regression: Bisected, IRQ and MSI allocations screwed with ..., Jesper Dangaard Brouer, (Thu Nov 20, 12:48 pm)
Re: Regression: Bisected, IRQ and MSI allocations screwed ..., Jesper Dangaard Brouer, (Fri Nov 21, 3:33 am)
Re: Regression: Bisected, IRQ and MSI allocations screwed ..., Jesper Dangaard Brouer, (Fri Nov 21, 12:35 pm)