| Siquijor Philips |
| Oct 7, 10:24 pm 2009 |
| Pyun YongHyeon |
| Oct 8, 10:45 am 2009 |
| rihad |
| Oct 7, 10:35 pm 2009 |
| Andrew Snow |
| Oct 7, 10:26 pm 2009 |
| rihad | Choosing two 10GiGE cards
Hi there,
We think it's time to switch from two GiGE bce cards to 10GiGE.
According to http://www.freebsd.org/releases/7.2R/hardware.html
the 10GiGE cards listed below are supported on amd64. Anyone has
personal experience using any of them them? Should I prefer drivers that
support DEVICE_POLLING (actually, only ixgb as per "man polling")? I'll
try to find out from the manufacturer how their own models differ from
each other (all this LR/SR/CX4 stuff). Thanks.
[i386,amd64] The ixgb(4) ...
| Oct 7, 10:23 pm 2009 |
| Andrew Snow |
| Oct 7, 10:42 pm 2009 |
| Jack Vogel | Re: Choosing two 10GiGE cards
If you really want to make the switch to 10G I would also seriously consider
moving to
FreeBSD 8, the changes that have been made in the stack will help you get
the most
out of the hardware.
When it comes to Intel hardware the model Andrew cites is the CX4 version of
the 82598, you
can also get it in fiber. The next gen mac, 82599, is out, but I am not sure
how easily they are
obtainable in the channels yet. My ixgbe driver supports both, the 599 will
have some advantages,
I am just about ...
| Oct 8, 12:18 am 2009 |
| Maxim Dounin | deadlock with pf uid rules + syncache
Hello!
We with ru@ investigated lockups our colleagues observing on
RELENG_7 machines, and was able to track [at least some of] them
down to the following deadlock:
1. syncache does syn+ack retransmit on timeout. it holds
tcp_sc_head mutex, and output packet processed by pf which in
turn waits for tcp lock to lookup uid.
2. em0 dispatches new ack packet to listen socket, tcp_input()
holds tcp lock, syncache lookup requires tcp_sc_head.
Here is data from ddb:
db> bt 19
Tracing ...
| Oct 7, 5:37 pm 2009 |
| rihad |
| Oct 8, 9:54 am 2009 |
| rihad | Re: dummynet dropping too many packets
2018 users online, 73 drops have just occurred.
p.s.: already 123 drops.
It will only get worse after some time.
Traffic load: 440-450 mbps.
top -HS:
last pid: 68314; load averages: 1.35, 1.22, 1.25
up 0+05:13:28 17:53:49
145 processes: 11 running, 118 sleeping, 16 waiting
CPU: 1.4% user, 0.0% nice, 2.8% system, 10.3% interrupt, 85.5% idle
Mem: 1337M Active, 1683M Inact, 355M Wired, 40K Cache, 214M Buf, 560M Free
Swap: 2048M Total, 2048M Free
PID ...
| Oct 8, 5:58 am 2009 |
| Julian Elischer |
| Oct 8, 9:18 am 2009 |
| rihad | Re: dummynet dropping too many packets
Been running for a few hours under these changed sysctls:
kern.clockrate: { hz = 4000, tick = 250, profhz = 4000, stathz = 129 }
net.inet.ip.dummynet.io_fast: 1
net.inet.ip.dummynet.hash_size: 512
net.isr.direct: 0
net.inet.ip.intr_queue_maxlen: 5000
kern.ipc.nmbclusters=111111
Current stats:
net.inet.ip.dummynet.search_steps: 190347891
net.inet.ip.dummynet.searches: 188979871
net.inet.ip.dummynet.io_pkt_drop: 0
net.inet.ip.intr_queue_drops: 0
Around 1800 entries in each of the two ipfw ...
| Oct 8, 5:14 am 2009 |
| Oleg Bulyzhin |
| Oct 7, 11:06 pm 2009 |
| rihad | Re: dummynet dropping too many packets
~4000 online users, ~450-470 mbps traffic, 300-600 global drops per
second. Same ole. Not funny at all.
net.inet.ip.dummynet.io_pkt_drop: 0
net.inet.ip.intr_queue_drops: 0
My last chance is to tweak the software transmit queue. Just how can I
alter its size?
P.S.: We're definitely going to buy a 10GigE card. Like this ...
| Oct 8, 8:32 am 2009 |
| Ian Smith | Re: dummynet dropping too many packets
On Wed, 7 Oct 2009, rihad wrote:
> Robert Watson wrote:
>
> > I would suggest making just the HZ -> 4000 change for now and see how it
> > goes.
> >
> OK, I will try testing HZ=4000 tomorrow morning, although I'm pretty sure
> there still will be some drops.
Even if there are, I'd like to know what (rough) percentage in increased
interrupt load you experience with HZ=4000 vs 1000 on that beast in your
application, or of any discernable effects on other running ...
| Oct 8, 4:19 am 2009 |
| rihad | Re: dummynet dropping too many packets
Besides having little (if any) positive effect on the output packet drop
rate, it runs pretty well, there's no apparent difference, no drop in
performance etc.
Current interrupt load snapshot as per systat -vmstat:
Interrupts
59606 total
atkbd0 1
ata0 irq14
931 mfi0 irq16
uhci0 uhci
4001 cpu0: time
23549 bce0 256
3118 bce1 257
3999 cpu3: time
3999 cpu2: time
4001 cpu1: time
4003 cpu4: time
4003 cpu5: time
4001 cpu6: ...
| Oct 8, 8:42 am 2009 |
| Julian Elischer | Re: dummynet dropping too many packets
you can not do anything about it if one of the custommers sends a
burst of 3000 udp packets at their maximum speed(or maybe some
combination of custommers to something which results in an aggreagate
burst rate like that.
In other words you may always continue to get moments when the pipe
releases a bunch of stuff that has a potential to over-run something
down stream.
Think of it as a dam in a stream...
If you have no dam, teh water level goes up and down gradually and by
small ...
| Oct 8, 9:45 am 2009 |
| rihad | Re: dummynet dropping too many packets
I've said this before, but: the PC is only dealing with the downstream
traffic, i.e. traffic arriving from the net. I won't say anything
against dummynet bursts overfilling hardware buffers, but they're
_only_taking place when the number of entries in the ipfw tables reaches
2000 or so, and the traffic load is at about 430-450 mbps. That is, it
_never_ happens before both conditions are true. Although raising HZ
from 1000 up to 4000 hasn't helped, which is somewhat contrary to the
idea of ...
| Oct 8, 10:04 am 2009 |
