“nfs send error 65″

Tags: , , ,
No Comments »

Proceeding with my Christmas presents, I have refactored my ${HOME} lab. One of the goals was to migrate my public gateway to a diskless Soekris Net6501 my beloved wife offered me :)

The overall PXE/NFS process is explained a billion times over the Internet, only particular point here is that I used dnsmasq instead of ISC DHCP.

Nevertheless I came across an issue that took me way too long to understand; while the boot process seemed perfectly fine, at some point, after Setting up ttys, init hung and the kernel showed the following message:

nfs send error 65 for

Looking at /var/run/rc.log I saw that the last rc script called was /etc/rc.d/pf_boot, which tries to read /etc/pf.boot.conf if it exists and then enables pf, therefore blocking all traffic. So that was only a firewalling issue. I created the /etc/pf.boot.conf file with the following content:

pass in log all keep state 
pass out log all keep state

which will be surcharged later by the real pf rules on the gateway.

Once done, init proceeded, and my new diskless / fanless NetBSD gateway was up and running.


I’ve unplugged that crappy net6501. Turns out Soekris provides 4 GBe NICs with a CPU which is unable to handle one single Gbps, not to talk about OpenVPN encrypted tunnels.
I kept the same NFS setup, but plugged a basic Core 2 Duo Intel CPU and a couple of GBe NICs to it, and they do the job perfectly.

Running snoopy on NetBSD

Tags: , , ,
No Comments »

Snoopy is a pretty cool piece of software that can log every exec(3) call to syslog. When it comes to security, that feature can be really handy.
Yesterday (Dec. 5), I commited security/snoopy to pkgsrc. The package comes with GNU/Linux related scripts in order to modify /etc/ld.so.preload so libsnoopy is loaded before libc and achieve its role. NetBSD doesn’t have a ld.so.preload file, instead, we use a flexible /etc/ld.so.conf configuration file which has the following syntax:

<library> <sysctl> <variable>[,...]:<library>[,...] ...

In our case, after having installed snoopy, you’ll just have to add the following line to /etc/ld.so.conf (or create it):

libc.so.12    kern.ostype    NetBSD:/usr/pkg/lib/libsnoopy.so,libc.so.12

Meaning that when kern.ostype sysctl(8) value is NetBSD (always true on NetBSD, obviously), libsnoopy.so will be loaded before libc.

Once done, /var/log/authlog will be filled with lines like:

Dec  6 09:36:46 coruscant snoopy[19394]: [uid:1000 sid:4525 tty:(none) cwd:/home/imil filename:/sbin/sysctl]: sysctl vm.loadavg
Dec  6 09:36:46 coruscant snoopy[29510]: [uid:1000 sid:4525 tty:(none) cwd:/home/imil filename:/usr/bin/cut]: cut -f2-4 -d 

Enable iSCSI support in NetBSD domU

Tags: , ,
No Comments »

Dynamic module loading via modload has a couple of issues with a NetBSD domU kernel, so it is not possible to modload iscsi.kmod.
In order to enable in-kernel iSCSI support, you’ll have to add the following lines to your kernel configuration and rebuild it:

pseudo-device   iscsi

scsibus* at scsi?
sd* at scsibus? target ? lun ?

dmesg should show this line:

iscsi: attached.  major = 203

You’ll then be able to start iscsid and manage your targets using iscsictl.

Migrating Debian Wheezy to LMDE

Tags: , , , ,
No Comments »

My “mediacenter”, a small x86 machine plugged to the living-room TV was a diskless (PXE/NFS root) Debian Wheezy until the past week end. After having tried Linux Mint on a laptop of mine and being impressed by its integration quality, I decided to migrate my mediacenter to LMDE.
I did not reinstalled the system, mainly because Mint does not support debootstrap, instead I followed a couple of HOWTOs I found on their forums: this one and this one.
Those HOWTOs gave the main matter to initiate the migration, but I’ve been bitten by a couple of dependencies issues. One mainly: udev; long story short, Wheezy is still build around udev 175, while Mint have switched to systemd-udev 204. The simple solution is to apt-get purge udev. Yes this is violent as it will wipe mostly everything from udev to anything X-related, but that’s a good way of mintizing the previous Wheezy packages.
After that, apt-get install mint-meta-debian-cinnamon mint-info-debian-cinnamon should do the trick and you’ll be able to use Mint’s beautiful environment: cinnamon.
Of course, depending on how much packages you’ve installed, the upgrade pain may vary.

Install NetBSD (or any PV-capable system) on IBM’s SoftLayer

Tags: , , , , ,
No Comments »

At ${DAYWORK}, I happen to use IBM’s cloud: SoftLayer. It has all the
features you’d expect from such a platform, and can instantiate pretty much any
major GNU/Linux distribution you’d think of; but here’s the thing, we also use
NetBSD for some infrastructure services, and as you’d guess, there’s no
NetBSD support at all on SoftLayer.

I had to reverse some bits of their provisioning system to understand how to
achieve NetBSD installation, but most of all, automatic provisioning.

Virtualization system?

First thing was to discover what hypervisor and which mode is used, PV? PV-HVM?

I must say their support was not really helpful, but hopefully a simple dmesg
gave pretty much all the informations:

[    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
[    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 0000000040000000 (usable)

Ok, Xen it is…

[    0.000000] Booting paravirtualized kernel on Xen

Oh well, PV then. Quite amusing since the support told me they used HVM
pretty much everywhere.

From now on, it is possible to think on some directions to take, but one
question remains, are we able to boot on anything else than the distribution
kernel? By default, the Debian GNU/Linux distribution I use as my playground
has 2 partitions, one for /boot and another for /; messing up with
/boot/grub/menu.lst (yes they’re using grub 0.97) confirmed I was able to
boot on any kernel I like. So I tried to boot on
/boot/netbsd-INSTALL_XEN3_DOMU and guess what: it worked. I was able to boot
the kernel and install NetBSD on /dev/xbd0f (seen as /dev/xvda2 on
Linux), which was Debian‘s root filesystem.


Now to the fun part. While it is possible to install NetBSD “by hand” with the
previous procedure, I really wanted to have that system provisioned
automatically, just like the Linux systems are.

Watching all the names used during the provisioning process, I deduced there was
some kind of scripting phase once the virtual machine has been instantiated, so
I followed the white rabbit.
First, following jawa‘s idea (friend and colleague of mine), I installed
snoopy to a newly created virtual machine, then created an image template from
it, and instantiated it: and bingo, /var/log/auth.log showed there was a
/root/install.sh that was called right after boot up. So I wrote a basic
init-script in order to dump /root content before it is removed by the
provisioning system, re-imaged the VM, re-instantiated it, and from then, I had
my eyes on their provisioning scripts. Nothing fancy, some tune2fs stuff,
networking setup, and all the variables I needed to prepare NetBSD :)

Obviously, I needed to keep the Debian partition in order to bootstrap the
whole process, but I didn’t need those 25G for that, so I booted in rescue mode
and used resize2fs / fdisk to shrink that partition to 1G, more than enough.

I installed a basic, unconfigured NetBSD system on a newly created partition,
seen as /dev/xbd0g under NetBSD and /dev/xvda3 under Linux.
On the Debian partition, I wrote the following script:

# cat /etc/init.d/test

[ ! -f /root/install.sh ] && exit 0
# backup original provisioning
tar zcvf /usr/local/root-dump.tgz /root
# install our new script
/bin/cp -f /usr/local/new_install.sh /root/install.sh

The new_install.sh script is a modified version of SL’s install.sh.
Basically, it has all the tune2fs stuff removed so NetBSD can mount the
ext2 partition without unsupported optional feature, and prepares all the
files needed by NetBSD on its first boot. I won’t paste SoftLayer’s parts as I
don’t know if this would break some terms of use:

# ...
# before this line are SL files sourcing


mkdir -p ${netbsd}

# generate an MD5 passwd for NetBSD
# YES I KNOW MD5 IS BAD, but it's the only common method between Linux and
# NetBSD, you'll have to change root password using the `passwd' afterwards
echo "root:${OS_PASSWORD}"|chpasswd -c MD5
grep ^root /etc/shadow|cut -f2 -d: >${netbsd}/rootpw
# change it back
echo "root:${OS_PASSWORD}"|chpasswd

echo "inet ${NETWORK_eth0_IP} netmask ${NETWORK_eth0_NETMASK} up" >${netbsd}/ifconfig.xennet0
echo "inet ${NETWORK_eth1_IP} netmask ${NETWORK_eth1_NETMASK} up" >${netbsd}/ifconfig.xennet1
echo ${NETWORK_eth1_GATEWAY} >${netbsd}/mygate
echo "net ${NETWORK_eth0_ROUTES} ${NETWORK_eth0_GATEWAY}" >${netbsd}/route.conf
echo "search whatever.com" >${netbsd}/resolv.conf
    echo "nameserver ${n}" >>${netbsd}/resolv.conf

# more SL stuff continues
# ...

# boot on NetBSD next time
cp -f /boot/grub/menu.lst /boot/grub/menu.lst.orig
cp -f /boot/grub/menu.lst.NetBSD /boot/grub/menu.lst
# uncomment this line only when you're sure of your template

Now that we have all the needed informations dumped, we can create a NetBSD
init script
that will use them to configure itself at first boot:

# cat /etc/rc.d/firstboot

# PROVIDE: firstboot
# REQUIRE: mountcritlocal
# BEFORE: network

[ ! -f /firstboot ] && exit 0

/bin/echo -n "SoftLayer provisionning from ext2 partition... "


/sbin/mount /dev/xbd0f /mnt

for file in ifconfig.xennet0 ifconfig.xennet1 mygate route.conf resolv.conf
    /bin/cp -f ${othernb}/${file} /etc/

/usr/sbin/usermod -p `/bin/cat ${othernb}/rootpw` root

/sbin/umount /mnt

/bin/rm -f /firstboot

/bin/echo "done."

Of course it is needed to have a /firstboot file on the root of your NetBSD
template, simply touch it.
In order to manipulate the UFS2 partition from Linux, you can use
fuse-ufs2 as described on the previous post.

Mounting UFS2 read/write on Linux

Tags: , , ,
No Comments »

I recently had the need to mount an UFS2 (NetBSD) partition under GNU/Linux, and while this is surprising, a standard Linux distro, Debian in my case, is not able to mount it in read/write mode.

I came across this project https://github.com/DanielO/fuse-ufs2 which has basic UFS2 read/write support. It is not very stable, I made it crash a couple of times while using vim on the mounted partition, but it does support simple operations like cp, rm and such.

Here’s how to build and use it under Debian Wheezy:

apt-get install build-essential fuse libfuse-dev autoconf automake libtool git libbsd-dev e2fslibs-dev 
git clone https://github.com/DanielO/fuse-ufs2
cd fuse-ufs2
vim fuse-ufs/do_fillstatbuf.c

Comment out the __st_ino. Do the same operation for fuse-ufs/op_readdir.c

make install
fuse-ufs /dev/xvda3 /mnt -o rw

Replace xvda3 with the actual partition you need to mount.
You can now access your UFS2 partition :)

WP Theme & Icons based on GlossyBlue by N.Design Studio
Banner from www.trynthlas.com
Entries RSS Comments RSS Log in
Optimization WordPress Plugins & Solutions by W3 EDGE