login
Search
Hello there.
We have a bunch of obsd firewalls, 8 at the moment, all working nice and
so forth. But we
need to add about another 4 in there for new connections and networks,
which means more
machines to find room for.
So basically I have been asked to investigate running all these
firewalls in two big boxes, with lots
of NIC's, with a bunch of openbsd vritual machines on them. One main box
for the primary firewalls,
one for the secondary. Each virtual machine getting its own physical NIC.
Personally I dont really like the idea, I can see things going wrong,
lots of stuff balancing on a
guest os and box.
Can someone please inform me if this is a really bad idea or not,
ideally with some nice reasoning?
Cheers,
Josh
What type of throughput is required between each segment? If you've been around here much, you've probably heard me espouse on the benefits of VLANs. This is certainly more elegant and secure than running a number of virtualized OpenBSD systems on non-OpenBSD virtual host. There's nothing wrong with running multiple firewalls where your physical topology dictates it. Virtualizing numerous firewalls in the same chassis just doesn't make sense. --- Jason Dixon DixonGroup Consulting http://www.dixongroup.net
Well, heck, if he's thinking of putting in lots of interfaces (probably to the tune of 1 interface per firewalled segment), why not just run ONE or TWO firewalls? Either vlan the things or dedicate one interface per network segment, both work well. Actually, use the two boxes, and carp them for failover. -- "This officer's men seem to follow him merely out of idle curiosity." -- Sandhurst officer cadet evaluation.
Because we have no idea what his requirements are. That's exactly why I asked for them. Obviously, CARP is good in any scenario, but it only provides redundancy. It has virtually nothing to do with his network design. --- Jason Dixon DixonGroup Consulting http://www.dixongroup.net
Read this: http://advosys.ca/viewpoints/2007/04/fuzzing-virtual-machines/ Read the paper linked there as well. Always good to go back to original source material. Anyone who told you VM technology and security had anything to do with each other was full of doo-doo. After reading that, I'd not want to put any "externally exposed" apps on a VM system. Granted, OpenBSD might not be the best entry point for a VM attack, but the foundation VM design is based on isn't as solid as people think. Nick.
I'll echo Nick's statements here. Virtualization does not provide reliable enough segmentation to rely on for security assurance. Do not buy into the market smack the vendors are putting out about it. As far as that goes, the more time goes on, the weaker the assumption of virtualized segmentation becomes. Research from IntelGuardians and other groups appears to be coming closer to completely unraveling virtualization security, at least in terms of how it's implemented in VMware for example. See also CVE-2007-0061, CVE-2007-0062, CVE-2007-0063, and CVE-2007-4496. DS
Virtual equals complex. Network devices are supposed to be reliable. Complex does not equal reliable - just ask anyone that has served in the military!
Ironically, today's ISC handler's diary entry talks to this as well. http://isc.sans.org/diary.html?storyid=3411&rss DS
Some commercial firewalls (i.e. Juniper/NetScreen ScreenOS-based gear) have been offering virtual-systems for years now. I think the negative comments received here may be appropriate when sharing the system with non-secure guest OSs, but it seems that it might be alright if its nothing but firewalls Cheers, Kent
I have no detailed knowledge of those devices, but I'm pretty sure their virtual firewalls are not accomplished by virtualizing the OS, but by supporting multiple routing tables and rules-sets. Virtual and virtual may be different in reality. /Tony
It sounds to me like the comments here are largely appropriate, virtualizing firewalls in the limited context that has been explained probably isn't a real good idea...at least due to perceived load. Additionally, if there are that many fireuwalls being ran, instead of numerous interfaces in a fewer number of machines, you're going to continue to have problems being able to virtualize enough hardware network interfaces. However, I don't fully agree with the sentiment that running a firewall in a virtual machine (let's be specific, VMWare ESX) guest environment. I'm running my firewall on a ESX 3.0.2 guest, and it works perfectly fine. That being said, you have to be aware of the VM configuraton. The majority of vulnerabilities in VMWare are patchable (so yes, someone needs to do maintenance), but are also issues that affect the VMKernel or service console, and with careful planning, the vulnerabilities can largely be prevented for being used as exploits on external interfaces. And one final note...although I am a fan of virtualization (I work for the company that owns VMWare), I really, really wish they did not have so many freaking patches...
(I'd hoped you would have prefaced that with a statement like "these are my stock options talking, but...") This is the kind of bad advice that virtualization companies (and naive users of those technologies) need to stop spreading. This security model is flawed, and people should not rely on these virtual machine environments to provide firewall services. Here's an entirely realistic scenario at this point: - Administrator pays loads of money for VMware ESX; for better ROI, he intends to replace several systems on the network with one big system running a number of VMs. Maybe there is a full DMZ (say, 10 hosts) on this box. One virtual machine is configured as a firewall, intended to provide packet filtering and other network security services for the other DMZ VMs. - A vulnerability is discovered that allows an attacker who has presence in one VM to execute arbitrary code on the host OS, or transfer files between guest and host. (Both of these have happened already. In fact, VMware Tools seems to be the perfect bit of flawed gateway software to make this even easier.) Virtualized segmentation is compromised at this point. - Attacker now has presence on host OS and can fully control all 10 of the VMs running on the host. VM segmentation was supposed to prevent this, remember? This includes the firewall which he can now play fun games with such as overwriting the ruleset. He can sniff network traffic for all the VM hosts since he has direct access to the host interface. In one short subversion, 10 (11) systems have been compromised through one flawed security model. A weakness in one VM becomes the thing that makes compromising all the others dramatically easier. Why subject your firewall to that? At least in a traditional non-virtualized firewall model, the attacker would have to pull out real exploits and attack real (secured) services to compromise the firewall, and it wouldn't fall at the same time as the other hosts. Yes, these kinds of of flaws have (so far) ...
When I provided patch support for Solaris 10, the number of times that a patch would not add to the global zone, thus affecting all local zones on the host, was fairly common. This affected airlines, banks and oil companies that you have heard of...... I know an OS is different to a VM platform, but you are still relying on someone else to do their bit. And just because you have a lottery ticket worth of support contract does not mean that it will actually get patched in a timely manner, from experience....
so what do you recommend? running all 10 services on the same non-virtualized machine?
I don't understand the logic of having multiple firewalls on one box. If one box can handle the throughput requirements of all the NICs, why not just one big firewall? Doug.
That's why god created competant network admins and NAT. -- "This officer's men seem to follow him merely out of idle curiosity." -- Sandhurst officer cadet evaluation.
We are talking about OpenBSD here, and support for VRF is not there. ciao Luca
That may change faster then you expect
These are great news. If the implementation will allow to assign interfaces to different VRFs it would solve the virtual router/firewall setup without the need for OS virtualization. ciao Luca
On 22.09-02:06, Luca Corti wrote: i have a feeling that the funds currently available for your virtualisation project would improve the quality and delivery of these requirements.
If I had such project and funds I'd certainly contribute. In the meantime I have assigned part of my limited resources to buying the CDs for the new release... ciao Luca
On 23.09-01:35, Luca Corti wrote:
my apologies, i had you confused with the original poster.
--
t
t
w
You are not always in control of all things. Powerful technology is about choice, not about one absolute right way. BTW, NAT sucks. ciao Luca
someone just needs to sit down and add the code to put interfaces into alternate routing tables and arp running there and you can have that on openbsd. ok, it is a bit of work (that I am not very interested in). but the hard part (introduction of multiple routing tables) is already done. -- Henning Brauer, hb@bsws.de, henning@openbsd.org BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam
Douglas A. Tutty wrote: There are lots of places where multiple firewalls are better than a single firewall. If one believed in the idea of "a perfect VM environment", it could make sense to do that. 1) Unrelated projects: If Project A and Project B are not related, keeping them on separate firewalls can simplify the rule sets and administration. 2) Separate administration: If you run a data center with lots of different people managing different systems, "They" can administer their systems without having access to (or messing with) "My" systems' firewall. When they screw up their rules, they don't break my systems (and I guess it works the other way, too. :) Note this has some cross-training benefit, too. I can be the Firewall Deity, but I do want to go on vacation. Fred may be a Firewall Jester, but with a bit of practice, he could possibly back me up very effectively. So, Fred manages a firewall for his projects, when he screws up, he learns lessons on a simple system, and when I am not there, he can babysit the "big" firewall, and if I get run over by a bus, he knows how to keep all the systems running. 3) Isolation: I had set up a firewall for a web app some time back. I had ZERO trust in the skills of the web developers, and even less for their security programming skills (and similar trust in my skills to audit their code). So, I stuck their app on its own firewall, completely isolated from our production environment. I also made sure that the various machines in the thing were each attached to their own leg of the firewall, so that we really had several layers of security between the Internet (bad guys) and the database (the valuable stuff). You would have to knock over Apache, then the app, then the DB to get to the data. Even then, they get to a DB Server which had ONLY THE BARE MINIMUM data required to accomplish the task at hand. If it wasn't for this design, you can be sure that database server would end up serving a lot of things as, $18k Oracle ...
Hi Nick. I understand your reasons. To me they look like reasons for separate firewalls on separate boxes. In the scenarios you mention, would you put separate firewalls on one machine? If I was going to put them all on one machine, I'd separate the administration of the box itself (me) from the people responsible for rule sub-sets. E.g. if one sub-firewall is dealing with traffic between NICs 1 & 2 (call it channel A), another between NICs 3 & 4 (call it channel B), I'd have the channels A and B admins submit rules sub-sets via rsync to the box. My script would then sanity check (ensure that they only dealt with the interfaces they were assigned) then incorporate all of them into a master rule-set that would then get tested and then put on-line. I would think that this, being only one firewall, would be simpler than several firewalls in VMs on one box; possibly more secure given the comments in this thread about the porus isolation between VMs. That's just how I would think of it. OTOH, I've never done any virtualization and never been into a proper data center. Doug.
Douglas A. Tutty wrote: That's where you are supposed to 1) recognize that my mysteriously mangled e-mail address is me and 2) Read back to my previous statement where I stated that I don't feel VM technology is suitable for externally exposed apps or security critical apps and 3) catch the implied sarcastic sneer in "If one believed in the idea of 'a perfect VM environment'" Yes, very separate is what I was recommending: no VM, keep them as separate as possible. When appropriate, of course. VMware and related technologies look cool, but it's an extra layer of complexity and security vulnerabilities. It is also a technology where the track record is "Coolness first, security when they catch us with our pants down". It is also something that is rarely done properly (for my definition of "properly"), but that's a different discussion for a different list. Nick.
Thanks Nick. I don't catch sarcastic sneer much in person, yet alone via email. Doug.
I don't like the idea of virtualiazing the firewalls either. It's just asking for trouble. What happens when the host OS gets hacked? Better I think to get some of these: http://www.netgate.com/product_info.php?cPath=67&products_id=369 and some soekris boards. You'll be able to fit 2 firewalls per 'u'. Then either use VLANS, or put a nic on each segment. -Bryan
Check out the HP c-Class BladeSystems offerings. It is sad that HP is marketing it with virtualization via Vmware. Just disregard the vmware affair.
I'm going to vote this idea as "really bad" for two reasons. Considering the technology out there at the moment, I'm going to guess you're looking at vmware. As everyone keeps saying, the security and reliability of your systems now inherits these characteristics from the VM host, and in some cases, the other guests. I personally can't trust the host after having worked on OpenBSD drivers (mpi and vic) inside virtual machines. In the mpi case I do know they took shortcuts in their emulation of that hardware which required extra tweaks in my driver. Even worse though, when I was working on vic I used to be able to crash the host by getting something wrong in the guest. I don't want a bug in one of the VMs bringing the whole lot down. Secondly, the performance of your guests aren't going to be as good as the performance of OpenBSD running directly on the same hardware. If you look at the path an arbitrary packet takes on the way out of an OpenBSD firewall its loosely something like (this path be accurate, but that's not the point): routing table, pf, enqueue, driver, hardware. If you look at that same packet in a VM: routing table, pf, enqueue, driver, virtual machine, vm input, vm routing, vm enqueue, vm driver, hardware. The amount of work per packet increases so the time taken to deal with a packet goes up, which means that the hardware can't deal with the same number of packets that a real OpenBSD install can deal with. Just my two cents, dlg
Hi Josh, VM is great and I use it a lot for test and development. It's a marvellous tool. I also think it's very good to make virtual web/ftp/whatever servers, it eases maintenance and add a layer of reliability. That being said it doesn't make sense to virtualize the whole network, especially firewalls, on the same machine. Your virtual OpenBSD is just going to be a fancy and heavy weight personal firewall. Personal firewalls are useful but they don't offer the same level of security. Your call. -- Die Gestalt
