FreeBSD networking issues: TCP offloading and checksum

In the past month, it’s the second time I’m being bitten by FreeBSD in the networking field.

First time with my own gateway, I had this weird behaviour where machines on a different VLAN than the main one would use the Internet at full speed but would struggle to make any transfer from the main VLAN.

Turns out this was a tcp segmentation offload issue, which seems to cause so much problems it is disabled by default in some appliances.

Simply add net.inet.tcp.tso=0 to /etc/sysctl.conf

or add -tso and -lro in rc.conf’s ifconfig_<interface>:

ifconfig_em0="DHCP -tso -lro"

Sources: here, here, and here

Yesterday I had a different issue, yet somewhat similar, on another network where the gateway is also FreeBSD, but this time on a virtual (kvm) machine, where LAN and WAN interfaces are of the virtio type, bridge and passthrough respectively. The gateway would let ICMP pass, but neither TCP nor UDP. This time, I had to disable more than tso, also rxcsum and txcsum:

ifconfig_vtnet2="DHCP -lro -tso -rxcsum -txcsum"
ifconfig_vtnet1="inet 192.168.1.254 netmask 255.255.255.0 -lro -tso -rxcsum -txcsum

From what we can read here it seems I also could have disabled those at kvm’s level.