What does CPU usage look like when this is happening? is there any other
resources that appear to be constrained?
J
On Wed, Nov 17, 2010 at 3:21 AM, RLW <serandom@o2.pl> wrote:
quoted text > W dniu 2010-11-16 16:14, Joel Sing pisze:
>
>> On Tuesday 16 November 2010, Robert Lewandowski wrote:
>>
>> Hello,
>>>
>>> PROBLEM: transfer speed is ONLY HALF if queue is defined in pf.conf
>>> although queue is 950Mbit (1000Mbit-5%)
>>> pf disabled: 768 Mbits/sec
>>> pf enabled, queue 950Mbit: 337 Mbits/sec
>>>
>>> ANALYSIS:
>>>
>>> - OpenBSD 4.8 default intallation.
>>> - Test made between OpenBSD 4.8 and Debian Linux.
>>> (between two Debian systems speed is more than 900Mbit/s)
>>>
>>> *********************************************************
>>> LAN interface: Intel PRO/1000 PT Desktop Adapter (PCIe, model:
>>> EXPI9300PTBLK)
>>> DMESG: em0 at pci1 dev 0 function 0 "Intel PRO/1000 PT (82572EI)" rev
>>> 0x06: apic 1 int 16 (irq 5), address 00:1b:21:05:1f:39
>>> *********************************************************
>>> Default settings of TCP window size:
>>> net.inet.tcp.recvspace=16384
>>> net.inet.tcp.sendspace=16384
>>> *********************************************************
>>>
>>> 1a) pf disabled
>>>
>>> root@router-test (/root)# iperf -i 1 -t 3 -c 10.0.0.6
>>> ------------------------------------------------------------
>>> Client connecting to 10.0.0.6, TCP port 5001
>>> TCP window size: 16.0 KByte (default)
>>> ------------------------------------------------------------
>>> [ 3] local 10.0.0.8 port 27600 connected with 10.0.0.6 port 5001
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 0.0- 1.0 sec 54.7 MBytes 459 Mbits/sec
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 1.0- 2.0 sec 54.7 MBytes 458 Mbits/sec
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 2.0- 3.0 sec 54.7 MBytes 459 Mbits/sec
>>>
>>> 1b) pf enabled, no queue
>>>
>>> root@router-test (/root)# iperf -i 1 -t 3 -c 10.0.0.6
>>> ------------------------------------------------------------
>>> Client connecting to 10.0.0.6, TCP port 5001
>>> TCP window size: 16.0 KByte (default)
>>> ------------------------------------------------------------
>>> [ 3] local 10.0.0.8 port 46912 connected with 10.0.0.6 port 5001
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 0.0- 1.0 sec 53.9 MBytes 452 Mbits/sec
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 1.0- 2.0 sec 52.6 MBytes 441 Mbits/sec
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 2.0- 3.0 sec 54.1 MBytes 454 Mbits/sec
>>>
>>> 1c) pf enabled, added queue to default pf.conf:
>>>
>>> altq on em0 cbq bandwidth 1Gb queue { q_lan }
>>> queue q_lan bandwidth 950Mb cbq (default)
>>>
>>> root@router-test (/root)# iperf -i 1 -t 3 -c 10.0.0.6
>>> ------------------------------------------------------------
>>> Client connecting to 10.0.0.6, TCP port 5001
>>> TCP window size: 16.0 KByte (default)
>>> ------------------------------------------------------------
>>> [ 3] local 10.0.0.8 port 38266 connected with 10.0.0.6 port 5001
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 0.0- 1.0 sec 33.9 MBytes 284 Mbits/sec
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 1.0- 2.0 sec 35.0 MBytes 294 Mbits/sec
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 2.0- 3.0 sec 35.8 MBytes 300 Mbits/sec
>>>
>>>
>>> *********************************************************
>>> TCP window size changed to 131072.
>>> net.inet.tcp.recvspace: 16384 -> 131072
>>> net.inet.tcp.sendspace: 16384 -> 131072
>>> *********************************************************
>>>
>>> 1a) pf disabled
>>>
>>> root@router-test (/root)# iperf -i 1 -t 3 -c 10.0.0.6
>>> ------------------------------------------------------------
>>> Client connecting to 10.0.0.6, TCP port 5001
>>> TCP window size: 128 KByte (default)
>>> ------------------------------------------------------------
>>> [ 3] local 10.0.0.8 port 32680 connected with 10.0.0.6 port 5001
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 0.0- 1.0 sec 91.5 MBytes 768 Mbits/sec
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 1.0- 2.0 sec 92.1 MBytes 773 Mbits/sec
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 2.0- 3.0 sec 91.2 MBytes 765 Mbits/sec
>>>
>>> 1b) pf enabled, no queue
>>>
>>> root@router-test (/root)# iperf -i 1 -t 3 -c 10.0.0.6
>>> ------------------------------------------------------------
>>> Client connecting to 10.0.0.6, TCP port 5001
>>> TCP window size: 128 KByte (default)
>>> ------------------------------------------------------------
>>> [ 3] local 10.0.0.8 port 41092 connected with 10.0.0.6 port 5001
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 0.0- 1.0 sec 80.5 MBytes 675 Mbits/sec
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 1.0- 2.0 sec 80.1 MBytes 672 Mbits/sec
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 2.0- 3.0 sec 80.2 MBytes 673 Mbits/sec
>>>
>>> 1c) pf enabled, added queue to default pf.conf:
>>>
>>> altq on em0 cbq bandwidth 1Gb queue { q_lan }
>>> queue q_lan bandwidth 950Mb cbq (default)
>>>
>>> root@router-test (/root)# iperf -i 1 -t 3 -c 10.0.0.6
>>> ------------------------------------------------------------
>>> Client connecting to 10.0.0.6, TCP port 5001
>>> TCP window size: 128 KByte (default)
>>> ------------------------------------------------------------
>>> [ 3] local 10.0.0.8 port 12499 connected with 10.0.0.6 port 5001
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 0.0- 1.0 sec 40.1 MBytes 337 Mbits/sec
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 1.0- 2.0 sec 40.1 MBytes 336 Mbits/sec
>>> [ ID] Interval Transfer Bandwidth
>>> [ 3] 2.0- 3.0 sec 40.0 MBytes 335 Mbits/sec
>>>
>>> *********************************************************
>>>
>>> any ideas, suggestions about this situation?
>>>
>>
>> The default length for a queue is 50 packets - this only allows you to
>> queue
>> around 75,000 bytes and the burstiness of TCP slow-start is likely to well
>> exceed this in your configuration (due to the BDP). I'd suggest increasing
>> the queue length - also run 'pfctl -vvs queue' or 'systat queue' and see
>> what's happening with regards to packets drops.
>>
>
> ok, a set qlimit to 200 and then 500, no difference
>
> queue root_em0 on em0 bandwidth 1Gb priority 0 qlimit 500 cbq( wrr root )
> {q_lan}
> [ pkts: 858820 bytes: 1300055838 dropped pkts: 0 bytes: 0 ]
> [ qlength: 0/500 borrows: 0 suspends: 0 ]
> [ measured: 29222.9 packets/s, 353.90Mb/s ]
> queue q_lan on em0 bandwidth 950Mb qlimit 500 cbq( borrow default )
> [ pkts: 858820 bytes: 1300055838 dropped pkts: 0 bytes: 0 ]
> [ qlength: 0/500 borrows: 0 suspends: 0 ]
> [ measured: 29222.9 packets/s, 353.90Mb/s ]
>
>
> best regards,
> RLW
