We have been using VMware vSphere in an environment where VM admins are not to be trusted as they are only customers who purchase resources from us. It might happen that they use their virtual machine and the assigned public IPv4 address to spoof network packets, share illegal materials over various P2P networks or attack other VM guests locally.
VMware offers no solutions for the above problems, the assumption is that ESX hosts are supposed to be inside of a protected corporate environment. Networking is done via virtual switches which only implement little network security.
What we need to prevent:
- MAC, IP spoofing
- Illegal P2P traffic
- Illegal SMTP traffic (spam relays, etc)
What we already have:
- Forged transmit protection: VMware knows the MAC addresses associated to the network interfaces of the VM, so it can drop packets with forged source addresses. (Doesn’t protect against IP spoofing)
What we did:
We created a small virtual machine where we installed OpenWrt, a small yet efficient Linux based firewall/router solution for embedded systems.
The VM parameters were:
Network interfaces: 2
The actual image only took 10MBs which already included every tool we needed.
Then we connected the interfaces between the VM and the outside network, and created a software bridge.
We ran tests on an ESX server (4.0) with a quad core E5420 Xeon CPU.
Initiating a P2P network traffic from the protected VM resulted in 100% CPU pull for the bridge VM, the network troughput couldn’t go any higher than 3.5MB/s. Increasing the CPU limit for the VM resulted in linear growth of network troughput. With 1Ghz we had 7MB/s, and without CPU limitation the CPU usage went up to 1.6Ghz maximizing our 100Mbit/s uplink. Turning on and off the firewall with many Layer2,3,7 rules didn’t seem to affect the throughput performance. At least we knew that the bottleneck was the CPU and the bridge.
The results were somewhat disappointing so we started to dig further.
Our first guess was that the 32bit Bridge VM didn’t use the HW Virtualization Technology in the CPU. To have VMware use the HW method we had to port openwrt to x86_64 CPU. The task was not easy and trivial but we managed to create an image. For our disappointment the results were nearly the same if not worse. Checking through pubications and papers we found that engineers at VMware made SW virtualization better than Intel-VT. URL
In almost every of their benchmarks the software method was faster. Funny thing for Intel.
We implemented our ideas into the Bridging Firewall VM, changed the port groups of the VMs to be protected and now they are spam, warez and dos-attack free.
Do you need a production ready application-level firewall in your vSphere environment similar to this one? Contact Us!
Posted 2010/04/29 00:02 by jos