A Post Most Significantly Devoid

….of any sensationalism whatsoever. Or of any content in any form. This post is just for a URL that one can see without a crushing feeling of disappointment overwhelming you when there is, in fact, no feat or image of epic proportions in said post.

Here, have a picture of some spaghetti:

(Mental note: set up category-based RSS feeds too)

[edit] P.S. I blame daffy

Convenience Services And Consumers

(this post is now a bit late to the party…)

From Light Reading:

“Steven Glapa, senior director of field marketing at the Wi-Fi offload vendor [Ruckus Wireless], says that most operators are at least exploring how to charge for Wi-Fi now. Most, like AT&T Inc. (NYSE: T), which has 29,000 hot spots, offer it free as a value-added service today. But Glapa says operators, in general, are considering bundling in an extra cost for the off-network access into data plans and counting that usage against the data cap. ”

Well, gee, imagine that. Can’t do boundless data plans without costing it in somewhere. I’m sure that lesson was somewhere in my high school economics class ~10 years ago, although the content would have been slightly different (likely a comparison of ice cubes in your drink would have applied). When playing in the infrastructure and/or edge services game, it’s hardly like things have no cost (or a very minimal one, at that) whatsoever. Perhaps this is a different scenario to the “cloud” space, where resources are extremely cheap and easy to come by. This is why some ISPs have gone with bundling services like these as VAS products on top of their normal offerings. An example of this would be M-Web in South Africa who bundles 500MB monthly free hotspot data into their packages, available for use at any Always-On hotspot in the country.

All in all, I’m somewhat surprised it took this long for people to realize this, but I hope that the kneejerk reactions from the suppliers can be controlled somewhat and that they rather come up with some moderately sane products to supplant internet access instead.

Oh, and can we please stop saying “free wifi” ever again? It’s a term that I’m almost convinced causes the death of baby unicorns each time it’s used.

IPConnect And You

The alternative title for this post would be “How your DSL connection actually works (if you’re in South Africa)”, but of course that’s silly long so we won’t go for that. This post is the one that was promised in a recent post, because it’s a subject that is often not entirely understood when being discussed, and I figured I’d rather get it done before I forget about it.

So, first things first: PPPoE. PPPoE is the protocol that’s used to dial up your session to your ISP. Okay, actually, I’ll have to back up a bit further than that. To summarise very shortly from Wikipedia, “Digital subscriber line (DSL) is a family of technologies that provides digital data transmission over the wires of a local telephone network”. The gist of this means “fast internet over your phoneline, while leaving the voice circuit free”. It accomplishes this is by sending specific digital signals over the same line at a higher frequency (this is why POTS filters are used, see more here), and these digital signals are often ethernet frames transported over an ATM circuit. This very last bit isn’t important to the layman reader, except to understand that in the configuration we have in South Africa, it’s not the ideal way to manage a connection.

Now there’s two ways one can normally work with this traffic when you, the customer, dials in. The first is how it currently works: you dial in from your computer, and Telkom “terminates” the session. What “terminates” means in this instance is that their systems are the peer that your communication speaks to (think tin-cans-with-string). The second instance (a scenario called Bitstream) is where your ISP would be the peer for your communication, and they would terminate the session on their LNS (L2TP Network Server). In either case, how this dialing works is by encapsulating a protocol called the Point-to-Point Protocol, or PPP, inside the ethernet frames (think school textbook with your notepapers pushed into the book at all the relevant pages). So effectively the PPP packets carry your actual data, with the ethernet bits being the boat for the river that is the Public Switched Telephone Network, or PSTN.

As mentioned in the previous paragraph, Telkom terminates the PPPoE session here. When you’re dialing in, their AAA servers get an access request for “alex@mydomain.co.za”, look up the AAA servers responsible for “mydomain.co.za” and sends off an access request with the same information, essentially asking “is alex@mydomain.co.za allowed to dial in with this information that was given?”. If your ISPs AAA servers respond “yes”, Telkom’s equipment will go ahead with attempting to set up your connection.

Here’s where it gets sticky. Because it’s Telkom’s network terminating the connection, there isn’t a lot of control that they can give over to the ISP on how to handle customer sessions on their equipment at a national scale, so instead they go for preconfiguring their routers with specific IP pools and routing information. This is why, if you dial from a certain line over and over again, you can quite possibly end up getting the same IP (because the device terminating your connection has a specific, finite set of IPs it could possibly give you). The configuration which Telkom uses for this is designed only around IPv4, and around their equipment “forwarding” your ISP your traffic once it has “de-encapsulated” it. Consequently, for various reasons (technical and otherwise), it is essentially impossible to deliver native IPv6 to an ADSL user in South Africa dialing up with PPPoE. This same configuration is also why all the static IP products in the market require a bit “extra” on top of just a normal dialing process.

The alternative configuration, Bitstream, is one where your ISP would be terminating all traffic, and could give you whatever services they are able to provide (within the constraints of their technical ability). Obviously the latter is the more desired option, and has been requested from Telkom for quite some time now.

Well, that’s it. I’ve tried to not go into an abundance of overly technical details in this post as I felt those could be better served elsewhere, but if there’s any questions or remarks, please do leave a comment so that I could look into it and attempt to answer or clarify.

Terminal-based Quicksearch

The title of this post might be a bit misleading, since it’s not about the history-search feature you often find in shells such as zsh, which is the shell I use. Incidentally, if you don’t know about this feature, try it out! See below:

Press ctrl+r and start typing out a partial command that you’ve used previously, you should see it pop up on your commandline, ready for use. In zsh, this is the history-incremental-search-backward feature on the line editor, which you can see more of over on this page.

But as mentioned, this post is about something else. Some time ago I saw Jonathan Hitchcock mention use of the open(1) command on OSX and thought this was pretty nifty, leading to me looking around for the equivalent on Linux. I came across ‘xdg-open’, which works with the freedesktop standards and thus generally respects your desktop-environment-of-choice’s application preferences. After using it a bit, I decided it was unwieldy (since there were too many commands starting with ‘xdg-‘), and aliased it to ‘xopen’, which has the benefit of being both short and easily tab-completable.

This has been working pretty well for me since then, and only recently did I come up with a slightly improved use of it. Every now and then I want to quickly check up something online, and I could certainly use lynx/elinks for this, but they’re also a bit painful to navigate with on many sites, so they’re not exactly ideal candidates. To the rescue comes my handy xopen alias!

function googsearch() {
  xopen "http://www.google.com/search?q=$*"
}

function googsearchphrase() {
  xopen "http://www.google.com/search?q=\"$*\""
}

Those are the functions I created, and they expand quite easily on my shell, suiting me on both laziness and versatility/speed. The end effect is they quickly fire off a query to google in my preferred browser, which can be one alt-tab away or focus by default (depending on your DE config). Later I *might* investigate using another search engine, but my typical use is on Google.

The only downside I can see to this is I can still only make it work on a local shell at this stage, so I’d have to see how I can make it work through ssh tunnels or somesuch. Maybe some sort of hack emulating a socket-forward as agent forwarding is done? If anyone has any ideas, please post them in the comments, I’d be glad to hear about it.

Update: just for clarity, what I meant with the last paragraph is that I’d want to call this command (or something to the same effect) on a remote server, and have the query executed on my local machine.

South African IPv6 Usage

Over the past while Simeon’s blog has had a few posts concerning IPv6, and this alongside a few other posts that I’ve come across essentially indicate a very sad state of IPv6 in South Africa.

A quick check on Sixxs shows that while there’s a whole lot of allocations, many aren’t seen on the internet at all. We (AS37105) have had our network fully IPv6-capable for quite some time and we’ve even tested native IPv6 connectivity (dual-stack and IPv6-only) delivered to the customer over iBurst‘s network on a PPPoE session, so with all this IPv6 and no-one to send packets to we started looking at who we could get online. We’ve had a pretty good relationship with JAWUG over the years, and as of last night we’re transiting a bit of best-effort IPv6 for them. One of our customers, SA Digital Villages, has also had an IPv6 allocation for some time and their transit is now IPv6-enabled as well.

Here’s to hoping for more IPv6 in SA soon!

 

P.S. In another post I’ll explain why it’s hard to get IPv6 to a Telkom DSL customer in South Africa natively.

Time, NTP and Shiny Things

I see that Regardt beat me to the punch on this one, but we recently got a Meinberg timeserver going. It’s stratum1, publicly accessible and speaks IPv6 fluently! We’ve added it to the pools, so if you use the poolservers you’re quite likely to end up on it sometime.

Zenoss – Find transforms

So I was looking around in one of my zenoss installs some time ago to find what EventClasses I’d set up transforms in, but didn’t feel like digging around through the entire tree of EventClasses (a cursory check now reveals that there’s 136 of them in my one installation). At the time, I solved the problem, extracted the data I needed, and then consequently forgot about it.

And then today I needed that info again. \o/ for IRC logs. To do this, connect to the dmd (on my system, which is installed with the debian package, the command for this is su -c “/usr/local/zenoss/zenoss/bin/zendmd” zenoss. Adjust it for your own system), and then run the following code

foo = dmd.Events.getSubEventClasses()
for i in foo:
    if len(i.transform) != 0: print "%s :: \n%s\n\n" % (i.getOrganizerName(), i.transform)

This will give you human-readable list of all your existing transforms, which makes it easy to find and re-use them.

Edit: this is confirmed working on 3.2.1 (and probably works on the rest of 3.x as well, post in the comments if it doesn’t). Thanks to jmp242 from #zenoss for testing.

Aptly named software

Just a quick post to also serve as a mental note for later, with two mentions to start off with:

  • RANCID — see here, aptly named because it is indeed quite rancid
  • slapd — the/a opensource LDAP server. The name is equally funny in Afrikaans and English, although the meanings are slightly different (“slap” would translate to “flimsy” for this specific context of amusement)
  • cacti – much like the cactus-plant from which this application derives its name (and which it uses as a logo), it’s guaranteed to be a pain to handle if you’re doing anything more than look at it
  • doxygen – check up the definition for ‘doxy’. This annoying software feels much the same way. Shacks up with something important (in this case the actual docs you care about), but you have to put up with it.

DNS Platform Migration Fun

This post could go by the alternative title “Screw you, ISC, and thanks for making software that makes me hate DNS even more”. So let’s dive right in, shall we?

(to those who don’t care for the intermediate ranting and DNS explanations, page down for the tech bits)

There are various criticisms of the Domain Name System — the thing which enables anything on the internet to turn “www.google.com” or any other such name into something that is meaningful to a computer (see here) — but for the most part it works reasonably well. You set up some DNS software, perhaps battle with the config for a while, and then it works. But as a quote I’ve seen somewhere (and can’t find the origin of now with a quick search) says, “you can’t truly recommend some software [tool] until you can tell me why it sucks.” And ISC’s BIND is arguably a highly irritating piece of software, which has over the years led to a rise in popularity for various other options. Amongst these you’ll find some general free/opensource implementations, as well as some commercial platforms:

(That’s the nice thing about diversity and openness — in this regard, an open protocol — you always get some choice and you can pick which one best suits your needs.)

 

Some years ago, long before my time at my current employer, there was a business requirement for some DNS support in our product suite. And BIND was chosen as the platform, since it’s a fairly well-known one. As time progresses, so do the things we do, and one day we found BIND was no longer sufficient to do what we needed to. Amongst others, things like a supermaster (a master from which a slave will accept all domain information, regardless of whether that slave knows of such a domain) and dynamic backend functionality were some of those needs.
Now some options like bind-dlz and friends existed, but none of these really suited us. In the end we decided upon PowerDNS with our own custom software written to handle the dynamic things as business rules would require, and set forth on this path. Some time passes with Rossi writing all the backend code which we’ve then successfully been running in combination with PowerDNS for some time now.

 

Of course, we still have all those old BIND-based installations to get upgraded, and this is that tale. Thankfully, the latest version of our platform was designed with exactly this sort of scenario in mind, since we have to inter-operate with other AXFR-speaking nameservers. So I think “let’s just use the config interface to add the migration host as a second slave, massage the data as required on there and then port that data over to the new platform” even as a tiny voice in my head says “it’s never that simple and you know it.” About 2 hours later I’m found at my desk swearing violently about all manner of things, which is my out when dealing with frustrating software. This is because I’d ended up trying to find out why BIND wasn’t actually slaving anything to my “new” nameserver, even though all the configs and zonefiles were right. Not just that, it had also at some point stopped slaving everything it should to the secondary nameserver, which at this point isn’t a worry since I’m replacing it anyway.

 

:: TECH ::

After figuring out the bits of the migration that matter — such as fixing up the SQL output (from the handy

1
zone2sql

tool from pdns) that had some oddities due to what looked like multiple $ORIGIN statements in one file — had been figured out, it was pretty painless to move. There were some fun points, like handling multiple $INCLUDE statements in a zonefile, and *hattip* to Jonathan Hitchcock (for the pre- and post-insert idea) and Bryn Divey (for googling better than I).

 

So, sed trick 1, splitting the file into parts:
1
2
3
cat foo.zone | sed -n '1,/match/p' > firstbit
cat foo.zone | sed -n '1,/match/!p' > secondbit
cat firstbit secondbit > newfoo
Sed trick 2, reading in an external file to use it as the replacement text. We have:
1
2
3
# grep INCLUDE 10_in-addr_arpa.zone
$INCLUDE "/var/cache/bind/10_in-addr_arpa.zone.ns";
$INCLUDE "/var/cache/bind/10_in-addr_arpa.zone.mx";
We do:
1
2
3
sed -i '/$INCLUDE.*\.ns.*$/ r 10_in-addr_arpa.zone.ns' 10_in-addr_arpa.zone
sed -i '/$INCLUDE.*\.mx.*$/ r 10_in-addr_arpa.zone.mx' 10_in-addr_arpa.zone
<code>

And tada, instant awesome. This reads the 10_in-addr_arpa.zone.mx file for us, and replaces from the appropriate “$INCLUDE” start to end with the contents of said file.

 

Another issue I ran into was having the generate the appropriate reverse-entry zones for all the public IP netblocks, and with two /21s and a /18 to worry about I wasn’t planning to do myself if I could help it, so I employed a quick hack with ipcalc and dnspython to transform my /18 into its various component /24s, and then generate reverses:
1
2
3
4
5
ipcalc 11.22.33.0/18 /24 | grep 'Network.*/24' | awk '{print $2}' | cut -d"/" -f 1
11.22.0.0
11.22.1.0
11.22.2.0
...
We can then easily manipulate these in python or sed or cut, depending on how hacky we feel, but I went with python since I was already using MySQLdb to insert the records after massaging them into the right form.
1
2
3
4
&gt;&gt;&gt; import dns.reversename
&gt;&gt;&gt; range = "10.22.0.0"
&gt;&gt;&gt; print dns.reversename.from_address(range).to_text().split(".",1)[1]
0.22.10.in-addr.arpa.
And that’s it for somewhat useful little tricks. There was a bit of a discussion had about delimited formats like this, and Piet Delport (see blogroll) hacked up a neat little delimited datatype which you can find over here. Quick usage instructions:
1
2
3
4
5
6
&gt;&gt;&gt; d = delimited('foo.bar.baz', '.'); d.sort(); print d
bar.baz.foo
d[1:] -&gt; 'bar.baz'
d[1:2] = ['x', 'y', 'z']; d -&gt; 'foo.x.y.z.baz'
&gt;&gt;&gt; d = delimited('0.2.1.10.in-addr.arpa', '.'); del d[0]; print d
2.1.10.in-addr.arpa.
And now as the sounds of Mogwai, Flunk and Placebo massage my tired noggin, it’s time for me to go to bed.

To the early adopters^Wclickers

I thank you for your quick clicking! Soon there will be more content here. Tonight I’m just painting the shed…