Install NetBSD (or any PV-capable system) on IBM's SoftLayer
by Emile `iMil' Heitor - 2014-11-03
At ${DAYWORK}, I happen to use IBM’s cloud: SoftLayer. It has all the features you’d expect from such a platform, and can instantiate pretty much any major GNU/Linux distribution you’d think of; but here’s the thing, we also use NetBSD for some infrastructure services, and as you’d guess, there’s no NetBSD support at all on SoftLayer.
I had to reverse some bits of their provisioning system to understand how to achieve NetBSD installation, but most of all, automatic provisioning.
Virtualization system?
First thing was to discover what hypervisor and which mode is used, PV? PV-HVM? HVM?
I must say their support was not really helpful, but hopefully a simple dmesg
gave pretty much all the informations:
Ok, Xen it is…
Oh well, PV
then. Quite amusing since the support told me they used HVM
pretty much everywhere.
From now on, it is possible to think on some directions to take, but one
question remains, are we able to boot on anything else than the distribution
kernel? By default, the Debian GNU/Linux distribution I use as my playground
has 2 partitions, one for /boot
and another for /
; messing up with
/boot/grub/menu.lst
(yes they’re using grub 0.97) confirmed I was able to
boot on any kernel I like. So I tried to boot on
/boot/netbsd-INSTALL_XEN3_DOMU
and guess what: it worked. I was able to boot
the kernel and install NetBSD on /dev/xbd0f
(seen as /dev/xvda2
on
Linux), which was Debian’s root
filesystem.
Provisioning
Now to the fun part. While it is possible to install NetBSD “by hand” with the previous procedure, I really wanted to have that system provisioned automatically, just like the Linux systems are.
Watching all the names used during the provisioning process, I deduced there was some kind of scripting phase once the virtual machine has been instantiated, so I followed the white rabbit.
First, following jawa’s idea (friend and colleague of mine), I installed
snoopy
to a newly created virtual machine, then created an image template from
it, and instantiated it: and bingo, /var/log/auth.log
showed there was a
/root/install.sh
that was called right after boot up. So I wrote a basic
init-script in order to dump /root
content before it is removed by the
provisioning system, re-imaged the VM, re-instantiated it, and from then, I had
my eyes on their provisioning scripts. Nothing fancy, some tune2fs
stuff,
networking setup, and all the variables I needed to prepare NetBSD :)
Obviously, I needed to keep the Debian partition in order to bootstrap the
whole process, but I didn’t need those 25G for that, so I booted in rescue mode
and used resize2fs / fdisk
to shrink that partition to 1G, more than enough.
I installed a basic, unconfigured NetBSD system on a newly created partition,
seen as /dev/xbd0g
under NetBSD and /dev/xvda3
under Linux.
On the Debian partition, I wrote the following script:
The new_install.sh
script is a modified version of SL’s install.sh
.
Basically, it has all the tune2fs
stuff removed so NetBSD can mount the
ext2 partition without unsupported optional feature
, and prepares all the
files needed by NetBSD on its first boot. I won’t paste SoftLayer’s parts as I
don’t know if this would break some terms of use:
Now that we have all the needed informations dumped, we can create a NetBSD init script that will use them to configure itself at first boot:
Of course it is needed to have a /firstboot
file on the root of your NetBSD
template, simply touch
it.
In order to manipulate the UFS2 partition from Linux, you can use fuse-ufs2 as described on the previous post.