Category Archives: tech

DHCP, LXC, phy-less (?) bridges, and checksums

tl;dr: if your lxc container on a bridged/veth network is randomly failing to get a dhcp address, you can probably fix it with `ethtool br0 x off  tx off` (br0 being the bridge interface).

With my home DSL acting up quite a bit lately (“lately” == month and a half now. “acting up” == ground fault, and waiting for the telco to fix it..), I’ve been doing a lot of sandboxing work on my microserver at home. But because of its resource scarcity (2GB RAM, and I just haven’t bought more yet), I’ve been giving LXC a go[0] (where I’d normally just do libvirt’d kvm). It’s pretty easy to get started (check this post for pretty much info you need), but I did deviate from the norm slightly.

I like my eth interfaces a server like this to be non-bridged, mostly because of a lack of ipmi (but also being brctl is a clown sometimes). So my setup for this at home looks like so:

auto lxc0
iface lxc0 inet static
  address 192.168.2.1
  broadcast 192.168.2.255
  netmask 255.255.255.0
  bridge_stp on
  pre-up /sbin/brctl addbr lxc0
  post-down /sbin/brctl delbr lxc0

I’ve also got dnsmasq listening on everything, although I hadn’t had it doing dhcp on that interface yet. Today I decided to change that, which is what led me to discovering this: (afaict) if you don’t have a phy interface attached to your bridge, checksum offloading behaviour on that bridge appears to be fucked-by-default.

How this manifested in my case was that lxc containers couldn’t succesfully DHCP (mostly silent failure), but doing a pcap or dhcpdump on the inside interface would show responses actually getting to your container. After some various derpery with dhclient’s options and applying enough patience, I finally managed to see a message: ‘5 bad udp checksums in 5 packets’. Some quick searching revealed people advising doing `iptables -A POSTROUTING -t mangle -p udp –dport bootpc -j CHECKSUM –checksum-fill` to fix this. Not just liking magic patches, I did check into why this is the case, and as mentioned earlier, it seems that this happens when you don’t have a phy attached to the bridge[0].

My network config for that interface now has a `post-up /sbin/ethtool tor0 rx off tx off` in it, and things seem dandy.

[0] – Mostly works pretty well. On debian wheezy you need some backported stuff for the various cgroups support. I think I got them from sid.
[1] – “seems”, because a) I don’t feel like testing this by attach a real phy to the bridge now, and b) I haven’t run into this before when using various other things (KVM usually) that were running on phy-attached bridges so I can only guess this is what happens. Feel free to test and let me know.

puppetvpn

Just a quick post to announce puppetvpn. It’s a puppet module for easily setting up openvpn links (p2p and hub-and-spoke), with the main point/attraction/difference being that it lets puppet take care of the key management effort.

Based on initial work by Tristan, so all credit as due goes there.

It’s not very smart right now (presumes a whole bunch of things). Patches accepted.

And another one

Seriously loving my cellphone company right now.

vodacom billing engine excellence

PRERATE ALL THE THINGS

So, to whichever unfortunate person ends up with my support ticket for this, I wish you luck.

P.S. I was told there’s more context needed, so: look at the bundle type in the top bar, and then at the used/avaiable counts. What happened (my guess, but I’m pretty sure) is that when I bought it, their rating system preburned all the data I’ve already done this month (under previous rollover bundle).

Rocket-solid .za DSL setup

After a couple of SOHO router iterations (not all my own, sometimes seen via other people), recurring problems:

  • overheating devices due to bad design
  • crappy stock firmware
  • bugs (that often never get fixed, or updates are never applied)
  • other assorted silly issues, like a 1000-connection session table, or small arp tables, or or or …

Much like the rest of the local linux community, I’d long just done DSL bridging and let a Linux box do the work of Real Internet Connection(tm), but that’s not the easiest pitch for Joe Average Home User. So I found an alternative set of things that works fairly well, and is solid enough for you to fire rockets at it. The recipe is as follows

  • 1x DSL bridge of some kind. Some Broadcom chipsets are excellent
    • I’ve had a hell of a lot of lifetime out of the DSL-2500U
    • if you want cheaper, this Tenda unit
  • 1x Non-shit router
    • A RB750 is pretty good for this. Gets your packets going without too much fancy. Has other benefits too
  • 1x Extra switch, because cheap gigabit is good
    • again, a Tenda unit
    • there are some cheap tp-links and HPs around too that I’ve had before

Cheap, effective, and just slightly annoying on amount of power sockets used. The power used is fairly low, too (we’ve had to test it on another project before, and it’s something like R20/month).

I’ll update the post a bit later with the basic tik config to apply.

My IRC setup

Potentially a somewhat bland topic, but I find myself referring to this often enough that I wanted to write it up; saves me the effort of explaining it in future.

First off, a couple of requirements:

  • low latency on the user input side
  • deal with my somewhat ridiculous volume of IRC usage
  • accessible from any reasonable platform (which I roughly classify as “anything with a keyboard for input, and has internet”)

So when I say “somewhat ridiculous”, that means:

  • 12 IRC networks
  • 65 channels (of varying volume)
  • varying numbers of query windows, usually about 30+ open

I flatten my jabber/gtalk to IRC as well, by using bitlbee. It counts among the 12.

Historically, I had this set up as irssi with irssi-proxy in the USA, then another irssi+irssi-proxy at my home (to join a network only accessible via the WUG at that point), and then I would connect my machine-local client to that server (which was on dyndns). A couple of pain points with this included the fact that dyndns sucks donkey balls, and syncing of logs (which I did with rsync at the time) was crappy. At the time I also had less IRC volume than I do now.

Aside from those pain points, and the occasional power outage at home (which just made me link up to my parent client), this worked well. Quassel’s backlog fetching is shiny, though. Very shiny. I wanted it. So I redid my IRC setup. Now there’s only one master server (currently in Germany), with a quassel core connecting to it. The irssi proxy config looks like so:

20:08:39 -!- Irssi: Module proxy/proxy already loaded
20:08:44 [irssiproxy]
20:08:44 irssiproxy_bind = 127.0.0.1
20:08:44 irssiproxy_password = passwordhere
20:08:44 irssiproxy_ports = freenode=6001 shadowfire=6002 oftc=6003 bitlbee=6004 ...

So basically:

  • /load proxy
  • /set irssiproxy_bind ip
  • /set irssiproxy_password ircpassword
  • /set irssiproxy_ports network=port network2=port2

Connect the quassel core up to the proxy, and that’s it. Infinite scrollback for any of my devices with quassel, and I can just ssh from some random server and connect up to screen as well (which has saved my bacon in DCs a few times).

And yes, I know this is (a bit) crazy.

USB port orientation usability idea

I’m not a designer by trade, so this is purely a quick image mockup. But imagine how much quantum turning could’ve been spared if this was in the standard from the getgo:

So what I’m thinking is that on the machine it could get indicated which side is which. And yeah, I finally got to post this thing, after meaning to do so (and continually forgetting) for a couple of months now.

Update: source for image original is Wikipedia

Something I really need to add to my system-prov script

root@likho:~# echo “blacklist pcspkr” > /etc/modprobe.d/diaf.conf
root@likho:~#

Poking at xkcd 1190

I’m probably not the only one doing this, but… let’s poke around at the innards of xkcd 1190. I’ll update this as I find things. I’m not particularly clued at JS yet (browsers are not my main playground), so I’m hitting this as it comes ;)

So far, useful things found:

  • there’s a minified script for making all of this happen, run it through a beautifier of your choice to be able to read it
  • append #verbose to the end of the URL for JS console logging messages
  • there appears to be an event listener thing going on, with what appear to be UUIDs attached to the events. I suspect they’re only format-similar though, because they must be time-dependent. Update: looks like it’s just kept in memory
  • the json passed to the xkcd servers looks like so:
s {type: "comic/time", data: "{"spread":5,"image":"a901246fd70dcd0054429bf55ced123ecead832300d73dedd78857d91eaff2df.png"}", lastEventId: "c0ddcdf0-9547-11e2-8001-1c6f659cb250"}

More as I find it

P.S.: Randall Munroe you are a hell of a nerdsniper.

“Quality of service”

Alternative title: what happens when you buy things that are licensed/run per TCP connection it can maintain.

hageshii% date; elegua; date
Fri Feb 15 22:08:41 SAST 2013
Linux elegua 2.6.32-5-amd64 #1 SMP Sun May 6 04:00:17 UTC 2012 x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
No mail.
Last login: Fri Feb 15 20:44:49 2013 from 41.10.98.194
elegua% Write failed: Broken pipe
Shared connection to elegua.za.net closed.
Fri Feb 15 22:18:47 SAST 2013

10 minutes almost to the dot and my connection is forcefully severed, presumably for inactivity. I wonder how many inadvertent breakages this can cause. It’s certainly annoying. Thanks, Vodacom.

(Yes, I know I can VPN around this, or use mosh, or or or. Unfortunately none of those were quick to do because I hadn’t booted this box in quite a while, and Expensive-with-expiring-bytes-G connection is better used on other things than this)

And this is what the trace looks like:

 Host                                                Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 192.168.43.1                                      0.0%   131    1.5   9.4   1.1 211.2  27.6
 2. 10.17.7.11                                        0.0%   131   54.6 327.0  38.5 5456. 844.4
 3. 10.242.249.2                                      0.8%   131   47.9 314.3  42.1 5400. 823.9
 4. 10.113.228.1                                      5.4%   131   48.8 309.2  42.4 5346. 832.0
 5. 41.192.248.18                                    12.3%   131   55.7 247.6  39.4 5290. 790.0
 6. vc-196-207-44-134.3g.vodacom.co.za                7.7%   131   52.8 294.0  39.4 5234. 815.3
 7. 41.0.4.1                                         10.0%   131   49.7 249.9  36.6 5178. 764.8
 8. 10.118.46.10                                      7.8%   130  423.0 474.4 210.7 5155. 851.3
 9. te-9-2.car5.London1.Level3.net                   34.1%   130  483.0 461.4 204.0 4123. 762.5
10. ae-52-52.csw2.London1.Level3.net                 24.0%   130  239.5 502.6 216.1 5009. 882.5
11. ae-57-222.ebr2.London1.Level3.net                24.0%   130  231.0 409.9 216.0 4010. 614.4
12. ae-22-22.ebr2.Frankfurt1.Level3.net              22.5%   130  237.8 473.7 216.9 5906. 897.6
13. ae-72-72.csw2.Frankfurt1.Level3.net              22.5%   130  250.2 445.7 219.9 5851. 793.7
    ae-92-92.csw4.Frankfurt1.Level3.net
14. ???
15. 195.16.162.254                                   32.6%   130  247.4 406.6 223.9 4726. 723.5
16. hos-bb2.juniper1.rz1.hetzner.de                  74.4%   130  233.1 740.9 225.0 5682. 1218.
17. hos-tr2.ex3k9.rz1.hetzner.de                     18.6%   130  339.2 491.0 221.3 5626. 841.9
18. elegua.za.net                                     1.6%   130  343.7 508.1 226.2 5571. 829.2

Hi-kwality packets.

Cellular data extortion(?)

With my DSL and everything (switch, RB750, DSL modem, HP Microserver) being struck by lightning this week, I’m presently using my 3G for a bit of access. Just to ensure I don’t trigger any massive out-of-bundle charges, I checked my remaining cap quickly so that I can then run a rough mental allocation of it all for the next while. Then I saw this:

“Hang on a minute,” I thought as I read the first block’s data values, “that should be closer to 1.7GB remaining.”

Then I scrolled down, saw the ‘forfeited’ counter, and began wondering whether these companies could be forced to stop making forfeiting part of their contracts. It’s truly not like this is a technical problem. This is a business decision they’ve made to let bandwidth you’ve bought artificially expire.

A quick bit of math: assuming I use ~300MB a month (this appears to be the general level of data I use, based on a quick checking of my phone’s stats), and that I’ve had this contract for 22 months now, and at the R268.99 I’ve been paying for the bundle each month:

  • (268.99/800)*500*22 = 3698.61

So that’s R3700 of “forfeit”, for no reason other than someone decided it’d be a good way to make money. And, as far as I know, all the operators in this country do this. For the less technical readers: as I mentioned before, there’s no technical reason this happens. It’s just an entry in some database, and can be updated. If anything, maintaining an expiry time on data probably leads to more technical issues than they’d otherwise have.

To compare, this would be like anyone buying up a bunch of things (toilet paper, toothpaste, whatever), and the storekeeper then removing it from your home if you haven’t used in soon enough.