Lessons learned with OpenVZ/Virtuozzo hosting companies.

I’ve run VPS systems based upon VServer, OpenVZ and Xen on my own equipment (see http://en.wikipedia.org/wiki/Virtual_private_server) primarily for security reasons. I’ve also run VMWare but that was more to maintain my sanity and control my Microsoft induced tourettes, rather than outright security.

If you trap an untrusted application inside a VPS the damage done to the host computer can be reduced.

In general I tend to rate the systems in order of decreasing desirability as Xen, OpenVZ and VServer. The rating is in terms of functionality maintained (ie just how close to a bare machine running an OS is the VPS in question). You tend to have more networking issues with VServer. OpenVZ has a little trouble with some of the more esoteric resource management issues and Xen seems just about perfect with the exception of troubles with proprietary drivers that aren’t Xen aware. Everything low level must be Xen aware.

In running downspout.ca inside a VPS run by someone else I found there are other issues you need to watch out for. In particular I first picked an OpenVZ/Virtuozzo based system. I was seduced by features offered by the VPS hosting company.

OpenVZ isn’t a Virtual Machine of any kind, it partitions an existing OS running on a machine and keeps all the virtual contexts isolated. It does this well. However all the virtual contexts are sharing the hardware context’s resource pool. VPS hosting companies don’t advertise all the resource limits that exist, just the big ones like memory, disk and bandwidth.

Specifically I was running Postfix as my MTA. It seemed to be running just fine until I was spammed. As soon as that happened postfix would either fall over or it would defer real email. In other words, a simple spamming was becoming a DOS attack on my email. With investigation I found that postfix was eating unix domain sockets for breakfast. To just start and sit at idle was on the order of 70 sockets. A single incoming message would push it higher. Two simultaneous messages was over 100 sockets. On a real system this is irrelevant, and I’d never even realised it could be a problem. It’s a real problem when your VPS hosting company restricts you to a total of 100 unix domain sockets, a number they don’t advertise. The limit got bumped to 300 but that didn’t really solve anything but the immediate problem because then the buffers associated with those sockets were still severely limited. Again all eaten by postfix.

I ultimately solved the problem by abandoning Postfix and using Exim. Exim fit the resource limits comfortably. It sure is nasty to work with though. And I could never trust it. I don’t think like exim wants me to, I think like postfix wants me to.

I started to look at other OpenVZ based hosting company forums. This resource limitation is definitely not unique to the company I picked. Much dissent exists.

I’m now on a Xen based system. You effectively have your own kernel and your resources come out of your own memory pool. I can now use postfix without trouble and receiving 3 spams in a row is no longer a DOS attack on my email.