Migrate FreeBSD root on UFS to ZFS

At ${DAYJOB} I’m using a FreeBSD workstation for quite a while. Everything goes smoothly except for the filesystem. When I first installed it, I chose UFS because FreeBSD installer said that root-on-ZFS was “experimental”. I later learned that nobody uses UFS anymore and that root-on-ZFS is perfectly stable. Thing is, I chose UFS and I deeply regret it. Not because of ZFS’s features that absolutely do not matter for me on the desktop, but because FreeBSD implementation of UFS is terribly, terribly slow when it comes to manipulate big files. When I say slow, I mean that pkg upgrade tends to FREEZE the entire machine while extracting archives. That slow. And before you ask, yes, there’s been a lot of tuning on that side.

So I got another hard drive and migrated the system over it.

In this memo, the disk to be root-on-ZFS is being seen as ada1 and the current UFS one as ada0. I first tried the “GPT / root on ZFS” method but later realized my machine couldn’t boot over it. So I felt back to “root on ZFS using FreeBSD-ZFS partition in a FreeBSD MBR Slice” method.

The commands listed below are all well documented in FreeBSD Wiki, in particular here: https://wiki.freebsd.org/RootOnZFS/ZFSBootPartition

gpart create -s mbr ada1
# align to 4096 bytes
gpart add -t freebsd -b 4032 ada1
gpart create -s BSD ada1s1
gpart set -a active -i 1 ada1
# I allocate 450G on a 500G disk
gpart add -s 450G -t freebsd-zfs ada1s1
gpart add -s 4G -t freebsd-swap ada1s1
zpool create zroot /dev/ada1s1a
zpool set bootfs=zroot zroot
gpart bootcode -b /boot/boot0 ada1
zpool export zroot
dd if=/boot/zfsboot of=/tmp/zfsboot1 count=1
gpart bootcode -b /tmp/zfsboot1 /dev/ada1s1
dd if=/boot/zfsboot of=/dev/ada1s1a skip=1 seek=1024
zpool import zroot
zfs set checksum=fletcher4 zroot

The simplest way of sync’ing the UFS disk to the ZFS disk is using rsync, here’s the “exclude-list” I used:

$ cat tmp/exclude-list.txt 
/dev/*
/proc/*
/sys/*
/tmp/*
/mnt/*
/media/*
/lost+found
/usr/ports/*
/usr/src/*
/usr/home/imil/games/*
/usr/home/imil/.PlayOnLinux/*
/usr/home/imil/.wine/*
/.amd_mnt
/zroot

And the rsync command, including the X and A flags in order to keep extended attributes and ACLs:

rsync -aAXv --delete --exclude-from 'tmp/exclude-list.txt' / /zroot/

Finally

echo 'zfs_load="YES"' > /zroot/boot/loader.conf
echo 'vfs.root.mountfrom="zfs:zroot"' >> /zroot/boot/loader.conf
# disable geom naming by diskid to avoid swapon errors
echo 'kern.geom.label.disk_ident.enable=0' >> /zroot/boot/loader.conf

and replace the swap line in /etc/fstab:

/dev/ada1s1b    none            swap    sw      0       0

I’m using that migrated disk as we speak, it’s blazing fast, no more freezes, plus I could add the old disk as a mirror using ZFS. Not bad after all.

Update

Here’s an article that explains the procedure to have a mirrored setup of the latter migration; no surprise, you’ll just have to follow the steps described before on a new disk and then attach both disks to a mirror vdev:

zpool attach zroot ada0s1a ada1s1a
zpool status -v
  pool: zroot
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
	continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Fri Apr 29 10:59:36 2016
        1.29G scanned out of 134G at 69.7M/s, 0h32m to go
        1.29G resilvered, 0.96% done
config:

	NAME         STATE     READ WRITE CKSUM
	zroot        ONLINE       0     0     0
	  mirror-0   ONLINE       0     0     0
	    ada0s1a  ONLINE       0     0     0
	    ada1s1a  ONLINE       0     0     0  (resilvering)