Install NetBSD (or any PV-capable system) on IBM's SoftLayer

At ${DAYWORK}, I happen to use IBM’s cloud: SoftLayer. It has all the features you’d expect from such a platform, and can instantiate pretty much any major GNU/Linux distribution you’d think of; but here’s the thing, we also use NetBSD for some infrastructure services, and as you’d guess, there’s no NetBSD support at all on SoftLayer.

I had to reverse some bits of their provisioning system to understand how to achieve NetBSD installation, but most of all, automatic provisioning.

Virtualization system?

First thing was to discover what hypervisor and which mode is used, PV? PV-HVM? HVM?

I must say their support was not really helpful, but hopefully a simple dmesg gave pretty much all the informations:

Ok, Xen it is…

Oh well, PV then. Quite amusing since the support told me they used HVM pretty much everywhere.

From now on, it is possible to think on some directions to take, but one question remains, are we able to boot on anything else than the distribution kernel? By default, the Debian GNU/Linux distribution I use as my playground has 2 partitions, one for /boot and another for /; messing up with /boot/grub/menu.lst (yes they’re using grub 0.97) confirmed I was able to boot on any kernel I like. So I tried to boot on /boot/netbsd-INSTALL_XEN3_DOMU and guess what: it worked. I was able to boot the kernel and install NetBSD on /dev/xbd0f (seen as /dev/xvda2 on Linux), which was Debian’s root filesystem.

Provisioning

Now to the fun part. While it is possible to install NetBSD “by hand” with the previous procedure, I really wanted to have that system provisioned automatically, just like the Linux systems are.

Watching all the names used during the provisioning process, I deduced there was some kind of scripting phase once the virtual machine has been instantiated, so I followed the white rabbit.

First, following jawa’s idea (friend and colleague of mine), I installed snoopy to a newly created virtual machine, then created an image template from it, and instantiated it: and bingo, /var/log/auth.log showed there was a /root/install.sh that was called right after boot up. So I wrote a basic init-script in order to dump /root content before it is removed by the provisioning system, re-imaged the VM, re-instantiated it, and from then, I had my eyes on their provisioning scripts. Nothing fancy, some tune2fs stuff, networking setup, and all the variables I needed to prepare NetBSD :)

Obviously, I needed to keep the Debian partition in order to bootstrap the whole process, but I didn’t need those 25G for that, so I booted in rescue mode and used resize2fs / fdisk to shrink that partition to 1G, more than enough.

I installed a basic, unconfigured NetBSD system on a newly created partition, seen as /dev/xbd0g under NetBSD and /dev/xvda3 under Linux.

On the Debian partition, I wrote the following script:

The new_install.sh script is a modified version of SL’s install.sh. Basically, it has all the tune2fs stuff removed so NetBSD can mount the ext2 partition without unsupported optional feature, and prepares all the files needed by NetBSD on its first boot. I won’t paste SoftLayer’s parts as I don’t know if this would break some terms of use:

Now that we have all the needed informations dumped, we can create a NetBSD init script that will use them to configure itself at first boot:

Of course it is needed to have a /firstboot file on the root of your NetBSD template, simply touch it.

In order to manipulate the UFS2 partition from Linux, you can use fuse-ufs2 as described on the previous post.