So anywhere I look for router performance on OpenBSD, all the benchmarks

are on small lines or old machines.  I also see mentions of people using

it in large scale installations, which is what I'm looking to do.  I

thought I'd ask here and see what people have done. 
I have 2 GigE lines from different providers balanced via BGP with full

routes from both providers.  Currently, these are running through a

Linux/Quagga/Iptables router/firewall with a P4 3.2 GHz.  The distro is

Gentoo, and we've stripped it down quite a bit.
We're pushing streaming video, so it's almost all outbound traffic by

about a 30:1 factor, and our average packet size is quite large - around

1200 bytes.  At the moment, when we hit about 350Mbps, the router gets

to ~30% CPU usage, and it appears that we stop being able to pass all

the traffic at full speed.  I don't see packet loss, but our traffic

graph flattens a good bit.  At those rates, we also start to see

crashing, but we haven't been able to figure out the exact cause of

those either. 
So, long story short, I need a new router.  We've looked at Cisco, etc.

and for what we're doing, it looks like we need a carrier class router.

I can get a decked out 12008 for about $8k, but I'd rather not spend

that much, or use the 2 feet of rack space.
I've used OpenBSD/PF for firewalls in the past, and loved them, so I'd

like to use it for a router if it can handle what we need.  Basically, I

need to be able to saturate both of those GigE lines.  I'm willing to

buy the brand-newest hardware - the PCI express bus should be able to do

2.5 Gbps, but I can't find anything that says I can push that much

through software.
I was also looking at the Intel I/O Accelerator, but I didn't see if

there was OpenBSD support for it.  I'm sure if there is, that would help

get me to be able to push the traffic I want to.
A long explanation, but I'm just hoping someone could give me some

insight here.
Alex Thurlow

Technical Director

Blastro, Inc...
[ message continues ]

I don't have the faintest clue about that kind of speed, and the old box

next to me would probably faint if showed these numbers. Still, some of

the stuff below, while tangential, might be useful.
OpenBGP, by any right, should not be a problem if you are not doing

anything grossly stupid (like trying to run this in 8 MB of memory). The

intel accelerator you mention is not supported, so that wouldn't help

any.
The one point I miss is failover capability; both the Cisco and OpenBSD

should be able to do this, but it's worth noting - and having.
		Joachim

I just wanted to ask this question to misc@. My situation is

100Mbps/100Mbps that is needed to be managed. I need bandwidth

management and I want to ask if someone has such experience. I plan to

implement it on OpenBSD. Any recommendations?
Shohrukh

Yes, please don't piggyback on unrelated threads.
		Joachim

well... "it depends".

we have a router at a customer that I have seen peaking above 750

MBit/s, and that was with relatively "mean" traffic (i. e. not all nice

big packets). so I'd say there is a realistic chance to get reasonably

close (and if everything else fails, you can still split outgoing over

two or so).

naturally, that requires somewhat carefully selected hardware, and

these are ones of the very few machines I run where we do not go for

GENERIC.* for a reason.
--

Henning Brauer, hb@bsws.de, henning@openbsd.org

BS Web Services, http://bsws.de

Full-Service ISP - Secure Hosting, Mail and DNS Services

Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam

What are the main changes you make to GENERIC Henning?
		---

		Liam J. Foy

		<liamjfoy@netbsd.org>

The issue as explain in the archive many times is not the level of

traffic, but the number of packets per seconds you pass and it's based

also on good network cards. Many can do in the 500mbps with their

OpenBSD router and more without to much issues. But again, what is the

limit is the pps, not the bps. S, if all your packets are in the 1200

bytes as you put here, you sure can test it with one OpenBSD and you

sure should have no issue with good decent hardware, but more

importantly, good network cards. That's really the key here.
I use it in public peering places no issues and I keep rolling out more

and more and my next one, as I go slow to be safe will be in Equinix

where I have close to 100 sessions and many full bgp feeds as well.
Test and adjust for your own needs, but you sure should be able to do

that better then your current setup. Funny that some replace their setup

with Cisco and I replace Cisco with OpenBSD as much as I can! My only

problem is really I can't replace Cisco DS3 and multi channel DS3 with

OpenBSD yet for the lack of decent hardware for that! (;<
But every Ethernet type are going away from Cisco one after the others

and hopefully before the end of the year, all will be gone!
Best,
Daniel

eotdm may be worth a look where you have both ends of the line.

some vendors mentioned here:
http://marc.10east.com/?l=cisco-nsp&m=117207521113785&w=2

Thanks, not really doing how I would like it.
But I was wondering however if it wouldn't be possible to use the 72xx

routers as dumb media converter?
Meaning, I have a few of them replaced by bgpd and using OpenBSD as a

more effective router.
I wonder how or if possible to actually configure the router to have all

traffic from/to the DS3 port to go directly to/from a Fast Ethernet on

that same router without the routing engine of that router to do

anything what so ever. Some other interfaces on that router could stay

the same and do as usual, etc. But pick for example two of them, one DS3

and one Fast Ethernet and configure them as a simple media converter if

you like. In on one interface out on the other and reverse regardless of

  what it is.
That would work well and allow to reuse old stuff put on the self now. (:>
Any idea if anyone have done something like this, or if that would even

be possible?
Using Cisco gear as dumb media converter for an OpenBSD driven network!

That would be pretty cool! Then a logo on it as:
OpenBSD power network!
That would be sweet.
Best,
Daniel

I don't think you can do this exactly, but you can run OSPF on them,

let OpenBSD handle the main BGP sessions, and feed back a small BGP

table to the cisco containing just the prefixes that it needs to know

how to route. Something like this...
physical: peer -> cisco -> openbgp

e-bgp:    peer <---------> openbgp  (n.b. multihop for ebgp sessions)

ibgp:             cisco <- openbgp
basically, cisco must know routes for any packets that will be fed

to it.
in some cases (e.g. one transit feed going into cisco) you may be

able to get away with just a static default route to the transit

on the cisco and OSPF or static routes back to your network.
Same with layer3 switches if you need more PPS than you can handle

on a PC and can live with limitations of the switches (e.g. restricted

table sizes and buffers).
I have ports for dynamips and dynagen if you need to play with cisco

configs and don't have spare ciscos: http://spacehopper.org/openbsd/

Oops, forgot that part.  At 325Mbps, we do about 60,000pps, so that puts

us at about 360,000pps needed for 2Gbps.

You'll have a hard time finding benches for that. To date, the best

reported is 150k pps which was on the intel E7520 chipset. That

was using em drivers. You're safest best for the most performance

possible would likely be using the intel 5000 chipset

(i.e. SuperMicro X7DB* motherboards) coupled with

SysKonnect SK-9S* line of network cards. Its probably

a safe bet that you'll be capable of 200K pps, but beyond

that is anyones guess.

Assuming correct choice of hardware can get you half way to the goal,

wouldn't it be an idea to buy two or more machines and use CARP

loadbalancing? Or isn't this possible when we are talking BGP?
Regards,

Martin

Yeah that's what I was thinking... you not only eliminate a single point

of failure, but you also split your pps throughput requirements in half.
Danno

Danno.appliedi.net/drupal/
-----Original Message-----

From: owner-misc@openbsd.org [mailto:owner-misc@openbsd.org] On Behalf

Of Martin Toft

Sent: Saturday, February 24, 2007 10:52 AM

To: misc@openbsd.org

Subject: Re: Router performance on OpenBSD and OpenBGPD
Assuming correct choice of hardware can get you half way to the goal,

wouldn't it be an idea to buy two or more machines and use CARP

loadbalancing? Or isn't this possible when we are talking BGP?
Regards,

Martin