Tag Archives: tech

Wasted Effort

So in South Africa, we have this weird microcosm effect where people feel it’s useful to redo things locally. It came about due to various factors, and there are sometimes some useful advantages (job creation when you’re dealing with products people could make by hand, for instance). The rest of the time, I just don’t see the point.

Case in point, MyBroadband recently started pushing some initiative called Gr8Signal. It’s purpose? To map out cellphone network signal info. “This sounds eerily like OpenSignalMaps“, some might say, so let’s compare their features quickly.


  • Signal heatmaps
  • Coverage info for multiple operators
  • Speed data
  • Comparison possible
  • “Average” rating


  • Signal blocks

Seriously, that’s it. You can’t compare different operators, or even use more than one at a time. Okay, so it’s a new project. Maybe it still needs to get some basic features. Let’s compare their interfaces.

OpenSignalMaps first:

I searched for “Johannesburg, South Africa”, and the first maps I get are an aggregate coverage heatmap for all available operators across all signal bands, the capability to filter based on a variety of options, and – if I click on the “Advanced view” (not shown here, but it’s that bar left left of the “NetworkRank™” section) – I get speed, latency, and reliability information.

Now let’s look at Gr8Signal.

And then another one, because I can’t select more than one operator at a time:

Yup, seriously. No ability to compare operators. The views are in blocks (…you guys do know how cellphone signal works, right? Fluid display is *far* better). The default map choice makes it ridiculously hard to see detail clearly. The overall feeling of the website is about as awesome as the SAWS page was. You know, that one where you could flatline your CPU by racing your mouse over the zooming menu icons.

Overall, I just don’t understand the point of this project. The data already exists in a much more useful form elsewhere, alongside wider coverage. So that’s about it. Let’s see what the MyBB guys have to say.

Update: I feel I should add that the reason I seem annoyed about this is the perpetual case of people just doing something “local” for, seemingly, no more reason than doing it “local”. If they brought something competitive to the table, or offered a better product, I’d totally support that. This very seldom seems to be the case. Why should we keep dishing out subpar products in the .za market? It just gives us a name as bad imitators. China 2.0.


So we survived the day pretty well. Yay for things going as they should ;)

A quick summary would be having one query regarding being unable to hit our test site and that turned out to be a browser issue at the client. The following counters from it (stats from around 15h00 SAST):

   2012-06-06  --  228 IPv4 only
   2012-06-06  --  5 Confused
   2012-06-06  --  1 Web Filter
   2012-06-06  --  46 Dual Stack - IPv6 Preferred
   2012-06-06  --  16 Dual Stack - IPv4 Preferred

Not bad, considering we only took it live sometime last night. Some other people didn’t get by quite so well on v6 day though. Yahoo was one of them. When trying to go to ‘www.yahoo.com’, we get redirected to ‘za.yahoo.com’ with the following DNS records:

vandali % host za.yahoo.com
za.yahoo.com is an alias for fd-fp2.wg1.b.yahoo.com.
fd-fp2.wg1.b.yahoo.com is an alias for ds-fp2.wg1.b.yahoo.com.
ds-fp2.wg1.b.yahoo.com is an alias for ds-any-fp2.wa1.b.yahoo.com.
ds-any-fp2.wa1.b.yahoo.com has address
ds-any-fp2.wa1.b.yahoo.com has IPv6 address 2a00:1288:f00e:1fe::3001
ds-any-fp2.wa1.b.yahoo.com has IPv6 address 2a00:1288:f006:1fe::3000
ds-any-fp2.wa1.b.yahoo.com has IPv6 address 2a00:1288:f006:1fe::3001
ds-any-fp2.wa1.b.yahoo.com has IPv6 address 2a00:1288:f00e:1fe::3000

This then blows up at one of their Accelerators:

Worth a slight thought, since Yahoo actually appears to see use over much of Africa.

All said and done, a fairly good day. Didn’t notice any major blowouts elsewhere in the internet (although I should note I wasn’t tracking all news), and I look forward to some write-ups by the usual people (Renesys, HE, Evilrouters, etc) in the next few days. We appear to remain one of the most well-connected IPv6 ISPs in South Africa, and in a pretty good position overall.

6th of the 6th, launch ALL the things

So, it’s World IPv6 Day^W^W^WIPv6 Launch Day (rebranding lulz). Go test your IPv6 at some appropriate site like here (hosted in South Africa) or here (somewhere else than South Africa, I didn’t check).

Google also had some conference announcement about the next stage in Maps. I wonder if it’s this thing I saw on Google Maps last night: Mapmaker. It appears to be 4sq meets Waze meets .. something. Let’s see at 9am PST.

Update: here’s a screenshot.

Second update: (fuck you WordPress editor) apparently Mapmaker’s been live since mid-April. Talk about a quiet launch.

Mapmaker Intro Screen


Hai, can I hav sum intergnats plox?

Alternative post title: IPv6 all up in hurr

Some years ago, before the age of cheap international access on local ISPs arrived here, dual-homing (or n-homing, depending on how pimp you were) on your residential connection was quite the fashion among .za tech-heads. But not the fancy sort with BGP and decent best-route selection, just a really grubby sort: two accounts, one local (as in .za routing table) and one international. You can read up about the full setup over here on Stefano’s site.

Due to the nature of the split, there was some fun. Fun in the order which things might come up, fun in which session’s routing is ready first, fun in DNS server overwriting, that sort of thing. Of course, I mean fun tongue-in-cheek, since it was mostly an annoyance. Especially when ddclient picks the wrong PPP session (“the config says ppp1, why are you using ppp0?”), or doesn’t want to ignore its cachefile (forcing you to wrap it in another script and delete the cache yourself), or when your line flaps and all pppd instances go into this weird race condition where they suddenly all acquire the same IP, or ….. well, I guess you get the idea. It was painful.

Thankfully times have progressed, and now it’s possible to get IPv6. Hell, if you’re in the right place you can even get a static allocation of v6. Working for AS37105, this is of course one of the work perks, since we (the tech team on the v6 deployment) dogfood it ourselves to make sure we know that things are actually working. Things we usually note are the following:

  • explosions in HE.net’s v6 core – hey, it happens
  • client apps misbehaving – surprisingly, chrome on my desktop is one of these
  • “mixed” support – mikrotik, for instance. you can telnet/ssh it on v6, but not winbox to a v6 address (I don’t recall if I’ve tested whether it connects if a hostname resolves to v6 address..mental note)

Personally, the best part for me is not having to ever deal with broken dyndns anymore, or having to maintain lots of funky NATs, or having to tunnel home and route traffic via the tunnel. If I just quickly want to ssh to my desktop, it has a hostname in DNS and it works. If I quickly want to check up on my traffic stats or anything else, I browse to yariman (my gateway/home store). It’s great, and makes my life that much nicer.

All of this said, World IPv6 Day next week! Are you all ready for your few days of carnage as other shitty ISPs run around unprepared? Bring on the future!

One other thing, props to PH.Fat for another good track. The track alone is cool enough for me to share it, but then I saw that the album (available on their website) is creative commons, and that just wins a bit harder. Nicely done, guys :)

P.S. Fuck you, WordPress content editor, and your stupidity in paragraph designation flow after bulletpoints.

Mikrotik and Rancid

In what’s probably turning into a series of posts detailing methods for dealing with Mikrotik routers from an operations standpoint, I’ve decided to do a bit of work detailing a bit more what I referred to in my previous post: the rancid integration for a Mikrotik router. I partly blame Lars Engström for all of this, as he recently got a Mikrotik RB1100AHx2 for their office, and has been asking me some questions about various little bits as he’s been going.

So here it is, my first ever set of commits/pushes to Github. Credit for most of the work goes to other people, linked from the readme in the repo; I just glued it all together in a nice-to-use fashion.

Oh, side mention: Lars also runs the aggregator of networking/tech blogs at this URL. It’s a pretty decent collection, and if you’re interested in networking, you might wish to follow it (for those of you who aren’t reading this post through it).

Update: as of about rancid 2.3.8, it appears that these patches have been merged into upstream. I didn’t really bother to check history, because the changelog has nothing in and I can’t be bothered to find where it was merged. One key difference is that the device type is called mikrotik (unlike mikrotik_{x86,rb}, in my setup), and presumably deals with the conditional commands by itself.

Screw you, Mikrotik

Particularly, your shitty scripting interface.

Using a lot of Mikrotik routers in various places, I’ve grown accustomed to the platform, and it really is quite flexible. However, it has its idiosyncracies. Among them are bugs and regressions between versions (c’mon guys, can you at least get some functional testing in place?), inconsistencies/inabilities in how some things are done at protocol level (unable to forward a default route in BGP), those sort of things. Now before I continue, let me say that I understand how such things can happen, but I do feel annoyed that they can take that long to get resolved. The lack of automated functional testing is also a major bugbear.

So, you have a router, and you probably care about its config. Most people who do this are familiar with the tool rancid (for better or worse – maybe at some point ranrod will be usable), and there are a set of patches to support mikrotik devices over here. As a side effect of this toolchain, I have a set of method that I can use to log in on many devices with a relatively low-effort command method. Consider the following scenario: someone leaves a company, and you wish to update passwords (in the case where you don’t have don’t have tacacs or radius backing auth). On a unix-like system, you have a few options, but in router-/switch-land you’re limited to some other things. Not to worry, we have shell loops, clogin/mtlogin, and a bit of ingenuity! Leading us to make this:

for location in list some locations here; do
  grep 'mikrotik.*up$' "$location/router.db"; done | cut -d':' -f 1 | while read line; do
  mtlogin -c ':global users;
              :foreach i in=[/user find where name="userinquestion"] do={:set users ($users . "," $i);};
              /user set numbers=$users password=shinymoonbicycles; quit' $line;

Seems sane enough. Start a for loop, parse our router.db files for a list of routers we care about, loop through those and run the following automated command sequence. Low-effort, quick, gets it all done. Except it runs into this issue:

[automation@Brain] > can't read "users": no such variable

Argh. Apparently we can’t declare or use variables in this fashion. Even though I can do that perfectly okay when I’m logged into an interactive session (…how is this determined?). “But hold on,” I think, “rancid uses +ct in the username to skip colour and terminal detection; maybe I can disable that to get it done”. And no, you can’t. QQ

In my mind, this sort of thing, on top of the earlier listed problems, the utter silliness of the scripting language (really, go look at the way I have to construct a list of users, or, well, anything), and other things such as no clear equivalent of the cisco-alike “no” command that can be used to negate/remove any statement in the config, are among the things that stand in the way of Mikrotik being taken more seriously.

Mikrotik, you have a product with pretty good potential. It wouldn’t hurt to improve these things a bit. You can sell craploads of non-wifi CPE equipment if you could win over the Cisco-hearts. Please, fix this crap.

*publishes post and mails the link to Mikrotik support*

On Clouds and Wavey Hands

A friend of mine, Jonathan, was recently busy investigating some web technologies for bootstrapping a project he’s starting on and during his search for easy non-local database alternatives he came across this post that compares offerings from Microsoft and Amazon. Upon reading the post, the following quote caught my eye:

“Not surprisingly, the access to SimpleDB is much faster from inside Amazon’s network than from outside (about three times as fast). Searching the web can lead to some references about how Amazon has optimized its internal networks so that internal access to the different services is much faster than outside access. To me, it seems as a form of lock-in, not a desirable feature, …”

I’ve ranted a bit about a lack of infrastructure understanding before, so even so I encounter something every now and then which leaves me impressed with how little people in general seem to care about how things work; or, otherwise put, with only caring that they work. I’m reminded of the one scene somewhere in the series of The Matrix movies:

Neo: Why don’t you tell me what’s on your mind, Councillor?
Councillor Harmann: There is so much in this world that I do not understand. See that machine? It has something to do with recycling our water supply. I have absolutely no idea how it works. But I do understand the reason for it to work.

Both parts of that statement hold true, and I feel that it’s the latter part that people sometimes miss out on. To bring my point back to the original excerpt, I feel it’s somewhat silly to point out the fact of higher latency access without indicating that you attempted to get an understanding of what causes this, especially if you then want to jump to the next point of saying “it feels like lock-in”. Certainly it’s true that Amazon would try to improve the offering within their network, as it just makes sense to bundle a good services experience, but there are factors to consider when using this sort of service from elsewhere, factors which influence things to varying degrees. The foremost I’d list among these is physics: it takes time for the digital to reach from one location to another, because there’s various forms of media conversion likely to take place (light-to-copper, copper-to-light), there’s routing and switching which needs to happen, there’s probably some service-specific firewalling, loadbalancing and application-server interfacing likely to happen. The list goes from these “run of the mill” items which you’ll encounter on a regular basis to other things such as TCP setup time (which can also influence things in various ways depending on a whole other set of factors).

On a bigger scale, this sort of almost cargo cult thinking is pervasive in various different areas of life, and a quote from Arthur C. Clarke comes to mind: “Any sufficiently advanced technology is indistinguishable from magic”. At the end of the day, I’m a big advocate for understanding how something works, as well as pushing boundaries and trying to improve things. So while I don’t think we should ever just sit back and be complacent with how things are, I do also think that we should strive to understand just that little bit more than we need to. I feel it’s usually, perhaps always, that extra little bit which puts us ahead of just “churning” and into the position of actually producing something just a little bit better.

Even though that little bit might not be much, a few hundred/thousand/hundred-thousand/… of it adds up. Hopefully I’m not just preaching to the choir with my current audience, but that someone else might also come across this post. And, as always, comments and thoughts welcome!

South African IPv6 Usage

Over the past while Simeon’s blog has had a few posts concerning IPv6, and this alongside a few other posts that I’ve come across essentially indicate a very sad state of IPv6 in South Africa.

A quick check on Sixxs shows that while there’s a whole lot of allocations, many aren’t seen on the internet at all. We (AS37105) have had our network fully IPv6-capable for quite some time and we’ve even tested native IPv6 connectivity (dual-stack and IPv6-only) delivered to the customer over iBurst‘s network on a PPPoE session, so with all this IPv6 and no-one to send packets to we started looking at who we could get online. We’ve had a pretty good relationship with JAWUG over the years, and as of last night we’re transiting a bit of best-effort IPv6 for them. One of our customers, SA Digital Villages, has also had an IPv6 allocation for some time and their transit is now IPv6-enabled as well.

Here’s to hoping for more IPv6 in SA soon!


P.S. In another post I’ll explain why it’s hard to get IPv6 to a Telkom DSL customer in South Africa natively.

Time, NTP and Shiny Things

I see that Regardt beat me to the punch on this one, but we recently got a Meinberg timeserver going. It’s stratum1, publicly accessible and speaks IPv6 fluently! We’ve added it to the pools, so if you use the poolservers you’re quite likely to end up on it sometime.

Zenoss – Find transforms

So I was looking around in one of my zenoss installs some time ago to find what EventClasses I’d set up transforms in, but didn’t feel like digging around through the entire tree of EventClasses (a cursory check now reveals that there’s 136 of them in my one installation). At the time, I solved the problem, extracted the data I needed, and then consequently forgot about it.

And then today I needed that info again. \o/ for IRC logs. To do this, connect to the dmd (on my system, which is installed with the debian package, the command for this is su -c “/usr/local/zenoss/zenoss/bin/zendmd” zenoss. Adjust it for your own system), and then run the following code

foo = dmd.Events.getSubEventClasses()
for i in foo:
    if len(i.transform) != 0: print "%s :: \n%s\n\n" % (i.getOrganizerName(), i.transform)

This will give you human-readable list of all your existing transforms, which makes it easy to find and re-use them.

Edit: this is confirmed working on 3.2.1 (and probably works on the rest of 3.x as well, post in the comments if it doesn’t). Thanks to jmp242 from #zenoss for testing.