login
Search
Hi misc, I'm currently looking for hardware alternatives for firewalls that should have more than four NICs. Currently we are buying R200s from Dell, but we have the 4 NIC limitation. We could tell Dell to install a quad port NIC (in addition to the two-port onboard card), but I haven't read good things about the way they work. I've also looked into soekris, but they don't seem to have enough CPU for what we want (this is pure speculation) as we also have intense IPSec traffic on some of these firewalls (I've seen that some of them could have encryption boards added to increase performance, but I don't know if it works for any kind of protocol, or at what rate). In any case, what I would like to have is firewalls with multiple NICs (at least 6 NICs) *and* sufficient CPU to let IPSec work alright at least at ~50Mbps (internal backbone firewalls). The multiple NICs are to use trunk, pfsync, real network interfaces, etc. Thanks, Martmn.
Why could you possibly need 6 physical interfaces? Even if you have a failover pair of firewalls and switches, with a dedicated pfsync interface, you could get by easily with three interfaces. The first two interfaces are trunked, one to each switch. Use vlan(4) interfaces with carp(4) on top of that. Your third interface would crossover between firewalls for private pfsync traffic. -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net/
Hmmmm. "Why would you ever want to do that?" - really not a good thing
to say to someone... Saying that means you lack respect for the person
or lack imagination. "What are you using them for" is a better response.
I've frequently used 5 ports on my firewall for multiple isolated subnets.
I've had very good luck with any of a number of 4-port cards. Unfortunately,
the good ones are no longer made. I'm using a 4-sf card which is available
on the surplus market for $40 or so. The sf chip occasionally stops transmitting
(maybe 2 or three times a week) but the driver (with the latest fixes) catches it.
The 4-dc card is better but harder to find.
Is there a requirement for low power or small form factor?
geoff steckel
That you frequently use 5 ports on your firewall shows a lack of respect for your switches, or a lack of imagination. -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net/
Wow... I've used 5 interfaces also, but for different internet links. Try do multi routing when you have lot's of different ip's of different ranges on the same if. Your pf rules will be a mess and, in some cases, it just does not work. Also, it is like we never heard of switch vulnerabilities allowing people on one vlan to see traffic of other vlans. Blindly trusting the switches is like being driven by a blind guy, it can crash every moment. I believe that there is a reason for everything, even using lots of network cards. Martin, i believe that using 4-port cards can have it benefits. Heard a lot of good things from the intel 4-port cards. Also, their performance isn't hit that hard, because the intel one s are pci-e. My regards, -- Giancarlo Razzolini http://lock.razzolini.adm.br Linux User 172199 Red Hat Certified Engineer no:804006389722501 Verify:https://www.redhat.com/certification/rhce/current/ Moleque Sem Conteudo Numero #002 OpenBSD Stable Ubuntu 8.04 Hardy Herom 4386 2A6F FFD4 4D5F 5842 6EA0 7ABE BBAB 9C0E 6B85
I knew it was a matter of time before the "vlan insecurity" bullshit hit the fan. RTFA. Who says anything about "blindly trusting" switches? If you can't correctly configure VLANs on your switches, and filter on vlan(4) interfaces in PF, you shouldn't be administering production networks. There's nothing functionally different between: $ext_if="em0" and $ext_if="vlan0" I've developed networks with over a dozen routed VLAN segments on a single physical GbE link. With carp(4) interfaces on top. It's easy. In fact, it's a hell of a lot less error- and failure-prone than managing 5 interfaces. If you're not going to use the features that came with those $5k switches you just bought, you might as well stick with $100 Netgears from Best Buy. -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net/
Yep. A few years ago when the "vlan insecurity bullshit" was all the rage we happened to be upgrading our LAN to gigabit. I was a bit leery from the experiences of dealing with Nortel's retarded (and proprietary) protocol-based VLAN crap. But I didn't want that to taint our future. So before deciding on a course of action (VLAN or physical separation) we picked up a couple of Cisco 2960G's, put them on my workbench and *BEAT THE FUCKING SHIT OUT OF THEM* trying all these VLAN hopping exploits that were talked about. Nothing seemed to work: the switches did their job. On our older Nortel 450's we did see some VLAN traffic leaking out when the things were flooded but those units dated back to the late 90's or so. Tech changes and improves. Fast forward and we've got these 2960G's everywhere, a couple of 3750G's doing the L3 work and feeding to the hardware out to the world. Nearly 20 VLANs going through various trunks (single gig and etherchannel). The stuff just works well when configured properly. Gord
Small clarification: we do have some physical separation. Our iSCSI traffic, SAN heartbeat and DMZs have their own VLANs and physical trunks. Previous message applied to all general user traffic. Gord
which is exactly the point. there are too many misconfigured VLAN setups out there, and some vendors (namely: cisco) have fucked up defaults. cisco (at least: used to, not sure about the current status, I long abondoned that crap) puts all ports in "dynamic" mode by default, where a port automagically goes to vlan tagged ("trunk" in their terminology) when they see their proprietary GVRP-alike protocol announcements, and worse, their "trunks" by default carry ALL !!! vlans. every other switch i came across has sane defaults as in ports do not automagically traverse to tagged and vlans have to be assigned to a port specifically, unless explicitely configered otherwise. also, averybody SHOULD have mac address limits on every port, VLANs or not. unfortunatly pretty much all vendors make that way too hard and have stupid limitations in their implementations, aka configurable mac address limit per port is 1-32 or unlimited (hello HP? stupid). all that said, I do trust PROPERLY CONFIGURED vlan setups. I do trust mine. I rely on VLANs and their seperation. -- Henning Brauer, hb@bsws.de, henning@openbsd.org BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam
I am curious and risk running off topic here, but... Henning, knowing that you run an ISP of sorts what type of routers are you using? I am curious the setup you have considering you've abandoned Cisco and apparently don't have high regards for HP. :) -- # Curt Micol
The bigger HP Procurve switches are ok. Some shit, as usual, but all in all very usable. Routers: OpenBSD, what else? -- Henning Brauer, hb@bsws.de, henning@openbsd.org BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam
Hi, Erm, and on the hardware side, please? Kind regards, --Toni++
5300XL specifically. The other bigger (more expensive) should be en par. the smaller ones like the 2650s or the older 2524 and the like pretty standard supermicros. dunno why people think that would be all so interesting. unless you do way over a hundred MBit/s (heck, make that over 500) any half decent server grade machine probably suffices. -- Henning Brauer, hb@bsws.de, henning@openbsd.org BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam
there is a 1u supermicro that has 4 onboard, on PCIe and PCI-X each. gives 12 ems in 1U. -- Henning Brauer, hb@bsws.de, henning@openbsd.org BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam
i see that people have already made this pointlessly heated, but i'll just put in my 2 cents nicely: unless you're routing ridiculous amounts of traffic, in which case openbsd might not be able to handle the pps count, it is probably best to trunk the four interfaces into the switch, put vlans and/or carp on top of that and not add a slough of extra interfaces. it's not for me to say that you don't need the extra interfaces but trunking and vlans will likely (1) save ports on your switches, (2) make your setup more resilient by having a larger number of interfaces for each link to fail through, (3) simplify the cabling and (4) minimize the number of switches required. btw, commercially available hw encryption accelerators are not very relevant anymore since there is so much idle cpu power in most modern machines. it's usually a better idea just to buy a faster machine or one with a cpu that does its own crypto acceleration, e.g. via C7. cheers,
On Fri, Jul 11, 2008 at 11:47 PM, Martmn Coco We run a pair of dell 1950s and have been generally happy with them. We run one dual port intel card and the two build in ports, no problem pushing about 400mbit. The intel cards have worked ok for us for years now in various versions. You can configure the box with two dual nics or two quad nics on the dell web.
Thanks! Have you tried the quad nics on those Dells? We do have a couple of R200s, 860s and 850s running with 2 dual port cards no problem, but we have never tried the quad ports.
Never done the quad in my maxchines. I havent heard anyone getting fired over it either though. A quick check on dells web indicates you have two pci-e slots in those r200s, why not get two dual nics. On Mon, Jul 14, 2008 at 8:28 PM, Martmn Coco
Hello, I do have around 20 Dell 860 and R200 with 2 cards Intel Quad ports. That is a total of 10 interfaces on those cheap Dell. You'll never hit any problem if you use only one Quad port. Be careful with 2 cards on 860. You'll have to order "Intel PRO/1000 PT Quad Port" and *NOT* the "Low profile" one. For the moment, no issues with them. We hadn't tested performance. These Dell protect small Internet link so we didn't bother check performance for links below 10Mb.
I run a pair of HP DL320 G5 boxes as a pair of failover gateways (pf/isakmpd/ospfd/dhcpd) and have an Intel Pro/1000 PT quad port card in each, giving me 6 interfaces. The onboard ethernet controller is bge, and the intel ones are em. I use the onboard for a crossover link between the two gateways, and then the other 4 connections are split into 2 bonded pairs. One is a plain old bond to a separate network and the other bonded pair has 5 VLANs running over it. Carp's used on all the links, pretty much, and it works great. I haven't performed any particularly scientific performance tests, but I did push ~800Mbit/s using iperf through them, from what I recall. If you were to stick two of the cards in, you'd need one full height and one low profile, as only one of the PCIe slots on the DL320 is full height. You'd also need to make sure you ordered the right version of the server (I think you can get it with one PCIe and one PCI-X slot as well as two PCIe slots). I'm still not sold on the benefits of bonding when you have a failover pair of gateways, but we had the budget for the extra ports, so why not? It gives me room to expand by breaking the bonds if necessary. Next task is to fix munin (or replace with something else) so that I can actually get bandwidth stats graphed. -- Russell Howe, IT Manager. <rhowe@bmtmarinerisk.com> BMT Marine & Offshore Surveys Ltd.
First of all, thanks to all of you that have replied. I've thought of adding VLANs, and will be doing it in the future maybe, but in our current situation, that's not possible; not all the switches support this option, and there's still some concern about security implications (specially in upper layers of the company). This may be unfounded, but there is not much that I can do for the time being, and keeping things "simple" by dividing networks physically does it for us right now. I know that it means more cables, more switches, etc., but we can also choose almost any kind of switch and do not need to manage each switch in addition to the firewalls. I really don't want to add to this discussion, but that's the way it's being done right now. Anyway, thanks to everyone!
Hi Gang, well heres my 3 cents, first why use a stupid PC (any os) for routing...... REALY BAD jue,jue brake down and buy a old Cisco 7200, 7500, 3600 they are all very good routers, I used a 7500 for a while and now use a 3640 i use pf as a transparent bridge behind my router.. and protects my servers I have 3 nics, (world, dmz, ssh) you could put up a firewall before your router and put everything out one vlan to the router. and I have a cisco 2900-xl-en switch with 3 vlans on it... and no bleeding.. enjoy -- View this message in context: http://www.nabble.com/Hardware-recommendation-for-firewalls-%28more-than-4-NI Cs%29-tp18413703p18899631.html Sent from the openbsd user - misc mailing list archive at Nabble.com.
Grab a Watchguard Firebox X off of ebay, they have 6 interfaces, and you can get them pretty cheap, some of the bigger ones have more, onboard crypto, perfect for building openbsd firewalls... you can run off a CF... I'm putting together a project that uses openbsd on these boxes. If you have any questions about running openbsd on them let me know: www.thewaffle.org Thanks, Jim
I just got some screenshots of the project up, if you care to take a look: http://www.thewaffle.org/screenshots.html There is also a working copy of the VMware image of the project availible for download, see the following for brief instructions on how to setup the image: http://www.thewaffle.org/Forum/viewtopic.php?f=11&t=11&p=16#p16 pardon the site design, not my forte, hopefully getting someone else to build me something better soon. Over the next couple days I'll get an image made for the WG firebox X series, I have one laying around that I can work on, hopefully by this weekend. J
It's nicer to look at this page: http://www.thewaffle.org/screenshots/ with this JavaScript bookmarklet: javascript:(function(){function%20I(u){var%20t=u.split('.'),e=t[t.length-1].toLowerCase();return%20{gif:1,jpg:1,jpeg:1,png:1,mng:1}[e]}function%20hE(s){return%20s.replace(/&/g,'&amp;').replace(/>/g,'&gt;').replace(/</g,'&lt;').replace(/"/g,'&quot;');}var%20q,h,i,z=open().document;z.write('<p>Images%20linked%20to%20by%20'+hE(location.href)+':</p><hr>');for(i=0;q=document.links[i];++i){h=q.href;if(h&&I(h))z.write('<p>'+q.innerHTML+'%20('+hE(h)+')<br><img%20src="'+hE(h)+'">');}z.close();})() I didn't write the bookmarklet, it's from https://www.squarefree.com/bookmarklets/ . regards, --ropers
all the series works ? the url doesn't work:The requested URL /alpha.html was not found on this server.
I just spent some time on this and got a working image for the Watchguard Firebox X 500-2500 platforms. For more info about it, I'm keeping track of everything in a forum here: http://www.thewaffle.org/Forum/viewforum.php?f=6&st=0&sk=t&sd=d&start=0 While I was at it, I pulled out an old Watchguard Firebox III and attempted to get the image working on it as well, to my surprise I was successful at this as well, tracking this platforms progress here: http://www.thewaffle.org/Forum/viewforum.php?f=25 These are great platforms for this application, onboard crypto accelerators and the 3port FBIII has a pci slot for expansion so you could get another 4 ports off it as well. They can be had for a reasonable price on eBay at most times. Let me know if anyon has any questions about this. Thanks, Jim
How odd. I know at least one site that runs all of their BGP off of OpenBGP on OpenBSD boxes that are dedicated as routers. In all cases, these systems outperform the equivalent Cisco hardware for a fraction of the cost.
Forget this. Cisco does CEF (cisco express forwarding) that's stream forwarding in hardware. You don't have a chance to reach this PPS with a pc / server based router (any os). And I don't think there is any equivalent hardware for Cisco and other router vendors. Because only routing decision is done in CPU / memory, packet forwarding is done on the "hardware layer"... so you can't compare Cisco CPU / memory against PC cpu / memory that's not fair :-) But software routers e.g. OpenBSD are cheap and work well. If you don't need more than about 800Mbit/s throughput and you want to save some money us software routers... but agree, with a good server hardware, intel nics, dual core cpu, etc. you can get good performance out off a server based router / firewall.
However, this only applies to best case traffic; the hardware path does not handle all possibile cases, and corner cases are shunted to the underpowered CPU for special handling. An attacker can take advantage of this and overwhelm a "hardware" router with far fewer packets than their marketing glossies would have you believe, so in order to get your desired performance in all situations you have to go with a much bigger system. One nice thing about "software" routers is that the spread between their best case and worst case performance is much narrower, so they are easier to size and test.
On the 3600, 7200, 2800, 1800 and everything else that is not a L3 switching router that costs over 100k everything is done in SW. Cisco CEF is nothing more then a fast path through the box that skips everything that is time consuming. It is still a software feature and everything runs over the CPU. Systems like the 7600 platform are able to do forwarding on the switch modules but unless you get the fucking expensive ones you have not enough cam space for a full feed. But it is not honest to compare a Cisco 7600 or other high end super expensive near line speed routers with a openbsd
My day job lets me "play" with "fucking expensive ones", I love that statement Claudio. If you want commercial hardware that handles large PPS rates you get purpose built hardware, not a Cisco router. I also support 100M feeds going through Soekris 5501 running OpenBSD and they perform very well. I also want to add in the comment that any IPv6 filtering is done in software, no matter who's box you get. diana
Ok, ok. What I said was what Cisco says :D And of course I meant the fucking expensive Routers. Don't get me wrong. I'm also using OpenBSD as router / firewall on server hardware and embedded on Soekris / WRAP. The performance is great. I just don't want to use PCs / BSD Boxes as area border routers, core routers, etc... Cisco hardware is much more reliable than PCs and the configuration is quite easy and structured. Configuring OpenBSD as a router is easy and structured as well, unlike Linux which is actually not structured :-) If you have the money buy Cisco Routers (or from similar vendors), if you have time and want to save some money use OpenBSD. bests
Sorry to hijack this thread slightly, but it's related I think: I'm looking to create an OpenBSD firewall/router for home. It's going to need to support two ADSL (UK, 8mbit) lines with PPPoA. And then a bunch (4) of f/eth ports, which is simple enough. Could anyone recommend any low-profile pci adsl models that'd work in this configuration with obsd? Thanks!
as in, lies, lies, lies. I can't second that. Cisco and good PC hardware are en par ime. The whole system, Cisco + IOS vs PC-Server + OpenBSD - the latter is ahead. Again, ymmv. I have had cisco routers crash upon typing "show what? that mess is nowhere near structured. It is not a config no. If you have the money get somebody clueful to set your OpenBSD routers up. If you actually do route amny Gigabit/s worth of traffic things get a bit complicated, you might have to go for juniper then. But cisco... pah humbug. -- Henning Brauer, hb@bsws.de, henning@openbsd.org BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam
NB: According to Wikipedia, Juniper's JUNOS OS is FreeBSD-derived. In other words, it ultimately evolved from the same ancestor OpenBSD evolved from. --ropers
On Wed, 13 Aug 2008, ropers wrote: So it runs some BSD derivative on it's management card, make no difference on how well the hardware and firmware is designed. Some Marconi ATM switches run Linux on their management cards, fortunately for us the hardware and firmware is designed well and they work. diana
rofl, if you think so... :) We should stop flaming about cisco vs opensource solutions... both have advantages and problems aswell. Its the common discussion about commercial vs opensource products, and this was not the idea when Martmn started this thread I think. My fault, sorry for that. bests
Careful now. CEF does speed things up in certain situations, but if
it's not backed by a very powerful cpu, you can easily completely
cripple your cisco by sending a stream of carefully crafted packets.
If you have to make a routing decision for every packet you process,
things will get nasty pretty fast. To handle such traffic, you'd need
even bigger boxes from Cisco while the OpenBSD solution does not care
all too much about this sort of thing (since it's not doing something
If you want more than 800Mbit/s you shouldn't use a 3600. With this
sort of bandwidth, you're going to have to spend a lot of money
anyway. Add to that the fact that the original poster was interested
in doing pfsync and ipsec on these machines, Cisco general purpose
routers wouldn't be a good match either.
Cheers,
Paul 'WEiRD' de Weerd
+++++++++++>-]<.>++[<------------>-]<+.--------------.[-]
http://www.weirdnet.nl/
1) that is best case. some traffic has to go to the main cpu. attackers can provole that and easily overload their tiny host cpus. 2) only the big models actually work that way. on everything 7200ish or smaller you have a classic central CPU design where almost all sure, it is fair. as long as an OpenBSD router is capable of handling your traffic (and I'd put the limit for real world traffic way above 800 MBit/s) it is, compared to cisco: -easier -way more secure -probably less troublesome and more reliable -way more flexible -cheaper. gives you money to donate. -- Henning Brauer, hb@bsws.de, henning@openbsd.org BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam
yeah, expect that it doesn't route everything and in the moment it falls back to cpu your router is dead. then there I saw all kind of "funny" and therefore extremely hard to trace and debug, bugs popping up with CEF well, up to around 500mbit any decent pc, doesn't even need to be server grade hardware will smoke any cisco, which costs >10 times more. if you need more performance, forget about cisco, get juniper if you really need something _fast_ or foundry. cisco only now brought some stuff to the market which comes close to what juniper delivered over the last years. will cost some money though. fast, reliable, cheap. pick two. ;) i wonder though how fast a nice openbsd machine with some 10g cards in PCIe slots will be. I guess we will soon find out, those things are getting "affordable". -sm
3600 are old and slow. They max out at 20Mbit/s at least the 3660 and 3640 we use are saturated easily. And getting a bgp full feed on them is just impossible now. 7200 are a bit better but unless you like to get the very expensive NPE-G1 card they max out around 200kpps and the backplane is plain PCI-32/33MHz so don't expect too much from the expansion cards. Again no luck with a bgp full feed I doubt you can fit more then 256MB RAM into the smaller CPU boards. The 7500 has it own issues manly the power consumption is insane for the capabilities of those beasts. Every remotely modern PC with PCI-E/PCI-X gigabit cards will smoke each and every of Using switches for fanout is great but remember most older 2900 cisco switches had a limit of 64 VLAN. Not a problem here but again a limit that I have run into causing unneccessary pain. Oh and getting IOS updates for used ciscos is another fun story unless you don't care about licensing.
You strongly overestimate the value of your comments (3 cents), it seems like there are many places more appropriate than this one for you to suggest middle-of-the-road hardware running a proprietary OS that has among the worst security records in the industry. -- Some software money can't buy. For everything else there's Micros~1.
Oh, god, Cisco vs <anyone else, especially free solutions> seems to degenerate into things like this. IOS and IOS XR actually has quite a good security history - other Cisco software, no. If you doubt me, actually look at the security record - oh, and be careful not to just compare OpenBSD's "only 2 remote holes in the default install" vs IOS - many (most) of the IOS vulnerabilities are for things that haven't been enabled by default on recent IOS images. Cisco routers general purpose computer parts of their routers are "middle-of-the-road hardware" in speed; much (slow) embedded hardware is far more reliable than the 'PC' equivelant. Server hardware (you shouldn't run anything important on a PC -- use proper server hardware) + Linux/Solaris/NetBSD/FreeBSD/OpenBSD works well as a router and firewall. IOS on a Cisco router does as well. The *nix solution works well and is cheap, but in my experience it's still slightly less stable than the Cisco equivelant. More importantly in many ways, Cisco hardware is usually marginally more reliable (both are reliable) than server hardware. IOS, while a complete PITA, is easier to configure than plain *nix OSes for networking stuff - one does not have sprawling config files, and making a config change updates running-config, making it easy to save your changes; ip address 192.0.2.0 255.255.255.255;do wr m is much easier than ifconfig fxp0 192.0.2.0/24;vi /etc/hostname.fxp0;<edit>. It's also much less error prone, which is important. With things like Quagga/Zebra this advantage is eliminated, but both of those have problems far more frequently than IOS. IOS is a lot easier to upgrade than any *nix - just copy the image, reload. Downtime is short, though many of their routers boot slow. This *could* be changed (I'm thinking something along the lines of Solaris LU - but easier), but as of yet has not been. But, it's *much* cheaper, and PF is vastly better than IOS's firewall. Software routers struggle at high PPS; Cisco makes some nice ...
