NetBSD as a Kubernetes Pod

I had to do it. So here’s how to run a NetBSD micro-vm as… a Kubernetes pod. First thing is to modify the start script from the previous article in order to add Docker-style networking, i.e. port forwarding from the host to the micro-vm. This is done using the hostfwd flag in qemu’s -netdev parameter #!/bin/sh kernel=$1 img=${2:-"root.img"} [ -n "$3" ] && drive2="-drive file=${3},if=virtio" qemu-system-x86_64 -enable-kvm -m 256 \ -kernel $kernel -append "console=com root=ld0a" \ -serial mon:stdio -display none \ -drive file=${img},if=virtio $drive2 \ -netdev user,id=net0,hostfwd=tcp::8080-:80 -device virtio-net,netdev=net0 In the previous experience we mapped the kernel and the root image from the host using Docker’s -v parameter, and while it’s possible to map files from the host using a Kubernetes volume, we will bundle NetBSD these files into the Docker image to make things easier.

NetBSD as a Docker Container

I have this little toy project for quite a while now, and I have this idea of handling a fleet of NetBSD micro-vms with Kubernetes since I started my new job in which I am caring a k8s cluster. I came to realize that starting a smolBSD micro-vm with Docker was not so difficult after all. Using mksmolnb’s I came up with this very simple Dockerfile: FROM alpine:latest RUN apk add --quiet --no-cache qemu-system-x86_64 iproute2 bridge-utils COPY startnb.

Fakecracker: NetBSD as a Function Based MicroVM

In November 2018 AWS published an Open Source tool called Firecracker, mostly a virtual machine monitor relying on KVM, a small sized Linux kernel, and a stripped down version of Qemu. What baffled me was the speed at which the virtual machine would fire up and run the service. The whole process is to be compared to a container, but safer, as it does not share the kernel nor any resource, it is a separate and dedicated virtual machine.

CPU temperature collectd report on NetBSD

pkgsrc’s collectd does not support the thermal plugin, so in order to publish thermal information I had to use the exec plugin: LoadPlugin exec # more plugins <Plugin exec> Exec "nobody:nogroup" "/home/imil/bin/" </Plugin> And write this simple script that reads CPUs temperature from NetBSD’s envstat command: $ cat bin/ #!/bin/sh hostname=$(hostname) interval=10 while : do envstat|awk '/cpu[0-9]/ {printf "%s %s\n",$1,$3}'|while read c t do echo "PUTVAL ${hostname}/temperature/temperature-zone${c#cpu} interval=${interval} N:${t%%.*}" done sleep ${interval} done I then send those values to an influxdb server:

5 minutes collectd + facette setup

I recently added a fantastic graphing tool named facette to pkgsrc. Facette knows how to pull data sources from various backends, and among them, the famous collectd. In this article, we will see how to setup both on NetBSD but keep in mind it should also work for any platform supported by pkgsrc. First up, collectd installation. It can be done either with pkgin (binary installation) or pkgsrc (source installation):

Simpler postfix + dspam

I have read a shitload of overcomplicated setups to bring up a postfix / dspam SMTP + antispam server, and finally came to a much lighter and simpler configuration by basically reading documentation and real life examples. Note this is suitable for a personnal and basic environment, no database, no virtual setup. Basic stuff. The target system is NetBSD but this short doc should apply to pretty much any UNIX / Linux.

NetBSD/amd64 7.0 on kvm

If you recently tried to install NetBSD 7.0 using Linux KVM you might have encountered the following failure: This bug have been recently fixed on the 7-branch but the official ISO images are not yet updated, so you’ll have to use NetBSD daily builds mini-ISO which includes Christos fix to bus_dma.c For the record, here’s the virt-install command I use: sudo virt-install --virt-type kvm --name korriban --ram 4096 --disk path=/dev/vms/korriban,bus=virtio --vcpus 2 --network bridge:br0,model=virtio --graphics vnc --accelerate --noautoconsole --cdrom /home/imil/iso/boot.

nfs send error 65

Proceeding with my Christmas presents, I have refactored my ${HOME} lab. One of the goals was to migrate my public gateway to a diskless Soekris Net6501 my beloved wife offered me :) The overall PXE/NFS process is explained a billion times over the Internet, only particular point here is that I used dnsmasq instead of ISC DHCP. Nevertheless I came across an issue that took me way too long to understand; while the boot process seemed perfectly fine, at some point, after Setting up ttys, init hung and the kernel showed the following message:

Running snoopy on NetBSD

Snoopy is a pretty cool piece of software that can log every exec(3) call to syslog. When it comes to security, that feature can be really handy. Yesterday (Dec. 5), I commited security/snoopy to pkgsrc. The package comes with GNU/Linux related scripts in order to modify /etc/ so libsnoopy is loaded before libc and achieve its role. NetBSD doesn’t have a file, instead, we use a flexible /etc/ configuration file which has the following syntax:

Enable iSCSI support in NetBSD domU

Dynamic module loading via modload has a couple of issues with a NetBSD domU kernel, so it is not possible to modload iscsi.kmod. In order to enable in-kernel iSCSI support, you’ll have to add the following lines to your kernel configuration and rebuild it: dmesg should show this line: You’ll then be able to start iscsid and manage your targets using iscsictl.