Monthly Archives: July 2014

DHCP, LXC, phy-less (?) bridges, and checksums

tl;dr: if your lxc container on a bridged/veth network is randomly failing to get a dhcp address, you can probably fix it with `ethtool br0 x off  tx off` (br0 being the bridge interface).

With my home DSL acting up quite a bit lately (“lately” == month and a half now. “acting up” == ground fault, and waiting for the telco to fix it..), I’ve been doing a lot of sandboxing work on my microserver at home. But because of its resource scarcity (2GB RAM, and I just haven’t bought more yet), I’ve been giving LXC a go[0] (where I’d normally just do libvirt’d kvm). It’s pretty easy to get started (check this post for pretty all much info you need), but I did deviate from the norm slightly.

I like my eth interfaces a server like this to be non-bridged, mostly because of a lack of ipmi (but also because brctl is a clown sometimes). So my setup for this at home looks like so:

auto lxc0
iface lxc0 inet static
  bridge_stp on
  pre-up /sbin/brctl addbr lxc0
  post-down /sbin/brctl delbr lxc0

I’ve also got dnsmasq listening on everything, although I hadn’t had it doing dhcp on that interface yet. Today I decided to change that, which is what led me to discovering this: (afaict) if you don’t have a phy interface attached to your bridge, checksum offloading behaviour on that bridge appears to be fucked-by-default.

How this manifested in my case was that lxc containers couldn’t succesfully DHCP (mostly silent failure), but doing a pcap or dhcpdump on the inside interface would show responses actually getting to your container. After some various derpery with dhclient’s options and applying enough patience, I finally managed to see a message: ‘5 bad udp checksums in 5 packets’. Some quick searching revealed people advising doing `iptables -A POSTROUTING -t mangle -p udp –dport bootpc -j CHECKSUM –checksum-fill` to fix this. Not just liking magic patches, I did check into why this is the case, and as mentioned earlier, it seems that this happens when you don’t have a phy attached to the bridge[0].

My network config for that interface now has a `post-up /sbin/ethtool tor0 rx off tx off` in it, and things seem dandy.

[0] – Mostly works pretty well. On debian wheezy you need some backported stuff for the various cgroups support. I think I got them from sid.
[1] – “seems”, because a) I don’t feel like testing this by attach a real phy to the bridge now, and b) I haven’t run into this before when using various other things (KVM usually) that were running on phy-attached bridges so I can only guess this is what happens. Feel free to test and let me know.


Just a quick post to announce puppetvpn. It’s a puppet module for easily setting up openvpn links (p2p and hub-and-spoke), with the main point/attraction/difference being that it lets puppet take care of the key management effort.

Based on initial work by Tristan, so all credit as due goes there.

It’s not very smart right now (presumes a whole bunch of things). Patches accepted.