Re: Bad network performance over 2Gbps

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Willy Tarreau <w@...>
Cc: Anton Titov <a.titov@...>, Linux Kernel Mailing List <linux-kernel@...>, <linux-net@...>, Jesse Brandeburg <jesse.brandeburg@...>
Date: Tuesday, April 15, 2008 - 4:40 pm

Willy Tarreau wrote:

# cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6
      CPU7
  0:        342        261        258        278        271        253        264
       283   IO-APIC-edge      timer
  1:          0          0          1          0          1          0          0
         0   IO-APIC-edge      i8042
  6:          0          1          0          1          0          0          1
         0   IO-APIC-edge      floppy
  9:          0          0          0          0          0          0          0
         0   IO-APIC-fasteoi   acpi
 12:          1          1          0          0          0          1          1
         0   IO-APIC-edge      i8042
 17:        180        190        178        183        182        186        186
       188   IO-APIC-fasteoi   uhci_hcd:usb1, ehci_hcd:usb4
 18:     843504     842514     843653     842033     842416     842742     841903
    842960   IO-APIC-fasteoi   3w-9xxx, uhci_hcd:usb3
 19:          0          0          0          0          0          0          0
         0   IO-APIC-fasteoi   uhci_hcd:usb2
498:  534642903  534635899  534726883  534732377  534701710  534708588  534730550
 534742730   PCI-MSI-edge      eth5
499:  531832274  531846609  531917849  531942676  531855140  531850692  531885565
 531863468   PCI-MSI-edge      eth4
500:  487251627  487279206  487248030  487220044  487239637  487231454  487281672
 487227202   PCI-MSI-edge      eth3
501:  486083953  486062203  486109925  486075793  486036977  486035152  486097551
 486117164   PCI-MSI-edge      eth2
502:  528889380  528863624  528760188  528798619  528891886  528890760  528807939
 528822746   PCI-MSI-edge      eth1
503:  529043135  529056706  528980250  528975209  529018995  529027386  528941583
 528970472   PCI-MSI-edge      eth0
NMI:          0          0          0          0          0          0          0
         0   Non-maskable interrupts
LOC:   62893699   62809502   62744208   62746035   62708815   62709055   62739182
  62620363   Local timer interrupts
RES:   15454866   15827970   16235695   15386970   15761053   16097167   16190851
  16159843   Rescheduling interrupts
CAL:         85         98         85         84         98         93         94
        91   function call interrupts
TLB:    3565361    3561798    3570271    3566272    3556996    3555866    3578257
   3564557   TLB shootdowns
TRM:          0          0          0          0          0          0          0
         0   Thermal event interrupts
THR:          0          0          0          0          0          0          0
         0   Threshold APIC interrupts
SPU:          0          0          0          0          0          0          0
         0   Spurious interrupts


Yikes! all wrong!

the network irq's are being ping-ponged around all the cores! bad!

1) turn the in-kernel IRQBALANCE option off !
2) use either the userspace `irqbalance` daemon or
3) set smp_affinity manually

Auke


--
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
Bad network performance over 2Gbps, Anton Titov, (Tue Apr 15, 2:06 pm)
Re: Bad network performance over 2Gbps, Willy Tarreau, (Tue Apr 15, 4:14 pm)
Re: Bad network performance over 2Gbps, Kok, Auke, (Tue Apr 15, 4:40 pm)
Re: Bad network performance over 2Gbps, Anton Titov, (Tue Apr 15, 6:36 pm)
Re: Bad network performance over 2Gbps, Willy Tarreau, (Wed Apr 16, 12:27 am)