Tales from the PinguCloud, part 1: NFS and WireGuard

[As I've mentioned before, I'm in the process of moving my various web sites from the dedicated server I've been renting with a few pals to the Hetzner Cloud. I'll be blogging about the experience here when (and if) exciting things happen. –A.]

After I'd first read through the Hetzner Cloud documentation, I laboured under the misconception that it would be possible to mount storage volumes on more than one cloud server at the same time (which would have been seriously convenient and cool). Alas, that's not how things work -- every storage volume is connected only to a single cloud server, which I discovered to my disappointment when I tried to put up a second CX21-CEPH server to take some of the load off the first one (Gitlab, in particular, seems to be the Brontosaurus of Docker containers). So what to do if the data on the storage volume has to be accessible from several servers?

I'm not particularly after HA right now and I'd rather do without the added complexity of cluster file systems at the moment. In any case, Hetzner claims that storage volumes are backed up three ways already, so the danger of losing stuff due to hardware failure seems fairly low. In fact, I have enough stuff on my plate right now without researching which of the 1,423 Linux cluster file systems (a) works, (b) is still maintained and available for the right distribution, and (c) is reasonably easy to install. Therefore the obvious choice for (at least) a short-term fix seems to be the old war-horse, NFS.

Now if you've been around the block a few times you know that NFS is a security nightmare and not at all what you would want to use in a public cloud environment where you have no idea who else is sniffing around your servers. But there are various ways of mitigating the exposure:

  • NFSv4 is actually a lot easier to deploy safely than previous versions. In particular, you can do without the dreaded rpcbind service (a.k.a. “The Portmapper”) which if left unsecured will get you nastygrams from the BSI. NFSv4 uses TCP with fixed ports and does away with a few other issues of the earlier versions. Also, it's been around for years and years and should be pretty safe to use from a maturity POV.

  • We'll be running NFS over a VPN and firewalling off all the relevant ports on the externally visible addresses.

Since I like to live dangerously, the VPN of choice for this deployment is WireGuard, which is the new very simple, very efficient kernel-based VPN that everyone is excited about. For Ubuntu LTS -- the distribution of choice for PinguCloud servers, since Docker Swarm is officially supported on it -- there is a PPA which contains the required WireGuard modules (neatly packaged using DKMS, which is not great but works, and is probably the only workable way of doing it until WireGuard enters the mainstream kernel).

Anyway, this combination looks reasonable to me under the circumstances and indeed it seems to work fine. In later installments of this series I may talk about the actual installation in a little more detail, so stay tuned.