Category Archives: minipost - Page 3

S(hitty)NMP

Posted by on 2012/07/11 No comments

This post will highlight Mikrotik/RouterOS issues, but it’s certainly not only them that suffer from S(hitty)NMP implementations.

Far be it from me thinking SNMP is perfect, nor that it’s necessarily always a good idea. I just have to wonder how it’s possible to screw up such a simple thing.

For example, Mikrotik has this nifty feature where you can look up the OIDs in a specific context by calling the print command with the parameter oid:

[user@R1] /system resource> pri oid
           uptime: .1.3.6.1.2.1.1.3.0
  total-hdd-space: .1.3.6.1.2.1.25.2.3.1.5.131073
   used-hdd-space: .1.3.6.1.2.1.25.2.3.1.6.131073

Except then this happens:

mon# snmpwalk -c ${com} -v 2c ${host} 1.3.6.1.2.1.25.2.3.1.5.131073
HOST-RESOURCES-MIB::hrStorageSize.131073 = No more variables left in this MIB View (It is past the end of the MIB tree)
mon# snmpwalk -c ${com} -v 2c ${host} 1.3.6.1.2.1.25.2.3.1.6.131073
HOST-RESOURCES-MIB::hrStorageUsed.131073 = No more variables left in this MIB View (It is past the end of the MIB tree)

The situation has at least improved vastly, though. Instead of finding the MIB file in some godforsaken dead corner of their documentation site (which is basically just kept on life support), the wiki has a formal section for it now. Still…things could be better:

  • Still no trap support in the various routing protocols/daemons (as far as I know)
  • Various bits of inconsistency like the above items
  • Indexes on dynamic interfaces and the like change, and with no way (that I’m aware of, once again) to lock them to a specific index irrespective of interface state.

There’s another issue that might’ve been fixed in the meantime, I haven’t checked in a while. SNMPv1 has a specific set of counter types, and anything bigger than n (where n was some signed integer limit or something) would only be displayable in SNMPv2. RouterOS just decided to not care about this at all, and respond with the number under the same counter type, but only ever when using SNMPv1.

Seriously, why is this stuff so broken?

HP iLO(2) tapdance

Posted by on 2012/07/11 No comments

Most people who run bigger sorts of servers are probably familiar with OOB management systems, but for those who aren’t here’s a short summary: you pay a little bit more when you buy your server, and you get a fantastical tool (vendors, please, get this stuff to fit the modern age. It’s not like we don’t want to use them) to use with your server. Power control, hardware status info, (usually) full IP KVM, etc. HP, Dell, Supermicro, Cisco UCS all have this in their own respective flavours.

That’s just to set the tone for what follows. So let’s pretend you live in .za, and you have crappy upstream bandwidth from your home. This would make things like firing up the HP SmartStart ISO on your hardware pretty painful, because uploading all that data takes forever. So what do we do?

We download it to another box on the same network and load up the image via a “hidden” section of iLO that allows us to mount images from an HTTP source, of course:

</>hpiLO-> show
status=0
status_tag=COMMAND COMPLETED

/
  Targets
    system1
    map1
  Properties
  Verbs
    cd version exit show

</>hpiLO-> cd /map1/oemhp_vm/cddr
status=0
status_tag=COMMAND COMPLETED

/map1/oemhp_vm/cddr

</map1/oemhp_vm/cddr>hpiLO-> show
status=0
status_tag=COMMAND COMPLETED

/map1/oemhp_vm/cddr
  Targets
  Properties
    oemhp_image=None
    oemhp_connect=No
    oemhp_boot=No_Boot
    oemhp_wp=No
    oemhp_applet_connected=No
  Verbs
    cd version exit show

</map1/oemhp_vm/cddr>hpiLO-> set oemhp_image=http://192.0.2.1/helpstuff/<ISO_Name_Here.iso>
status=0
status_tag=COMMAND COMPLETED

</map1/oemhp_vm/cddr>hpiLO-> set oemhp_boot=Connect
status=0
status_tag=COMMAND COMPLETED

</map1/oemhp_vm/cddr>hpiLO-> show
status=0
status_tag=COMMAND COMPLETED

/map1/oemhp_vm/cddr
  Targets
  Properties
    oemhp_image=http://192.0.2.1/helpstuff/<ISO_Name_Here.iso>
    oemhp_connect=Yes
    oemhp_boot=Always
    oemhp_wp=Yes
    oemhp_applet_connected=No
  Verbs
    cd version exit show

So, in summary:

We cd to the path that contains cddr (which is the virtual disc path). A note on this, the vm path might sometimes be oemhp_vm1. Do a show under /map if you can’t find the thing.
Then we set oemhp_image and oemhp_boot to values useful for booting.
Now we reboot.

After you’re done with stuff, just set oemhp_boot to Never, and it’ll disconnect stuff.

I didn’t check whether this worked for iLO3 as well, but I’d guess it’s relatively similar. Been a few months since I even looked at an iLO3 system. Here’s the command ref doc for iLO2 if you want to dig around for some more cool stuff.

Timejumps

Posted by on 2012/06/30 No comments

So today/tonight/sometime is leap second day. I’m not too sure when it is, exactly. Why? Because I don’t need to:

Jun 30 14:43:09 stratum1 lantime[1850]: Normal Operation  
Jun 30 14:43:17 stratum1 lantime[1850]: Leap second announced  
Jun 30 14:44:12 stratum1 ntpd[2172]: synchronized to PPS(0), stratum 0
Jun 30 14:44:13 stratum1 lantime[1467]: NTP sync to PPS

My timeserver knows. Firmware updates applied, leap seconds announced, music festivals to go to.

Mineshafts

Posted by on 2012/06/25 No comments

Or: when you seriously need to tunnel

I’ve got some servers sitting 300~500ms away, behind a bad NAT, and GRE/pptp can’t make it through. Quick way to solve it? Build a small crappy VM, install ssh, and make the following modifications to files:

/etc/ssh/sshd_config: append the PermitTunnel directive. Pick one you like from `man 5 sshd_config`
/etc/ssh/ssh_config: append the Tunnel directive. Again, check which you want from `man 5 ssh_config`.

Quickly generate a key for use for the tunnel dial and push it to your dial host:
ssh-keygen -C “tunneling key” -t rsa -f ~/.ssh/tunnel_rsa
ssh-copy-id -i ~/.ssh/tunnel_rsa user@tunnelhost

Now start up the tunnel:
ssh -NTCf -w any user@tunnelhost

Slap IPs on each side:
client:~# ip addr add 192.0.2.1/32 peer 192.0.2.2 dev <tundev>
tunnelhost:~# ip addr add 192.0.2.2/32 peer 192.0.2.1 dev <tundev>

Also, I noticed that between two debian hosts the tunnels defaulted to state DOWN, so a quick ip link set up dev <tundev> was needed each side.

Ping across, check if it works, and if all’s good you should be able to route via the tunnel and do whatever you need to. Since ssh is generally pretty capable and usable everywhere (even over some crazy portforwards), this should get you going fairly easily.

Crescendo!

Posted by on 2012/06/25 No comments

So I decided to, instead of spamming people up through my blog and IRC and jabber and …. each time I find something cool, rather make a concentrated little project for it.

And I had a useful domain for it around from 2010 as well. So, presenting Earnoms!

Check out the about page for a summary of the project, but definitely keep those music links coming :)

Musical Interlude

Posted by on 2012/06/20 No comments

After a few days of flu-sourced incapacitation, I’m back onto sorta being alive. Here’s a nice chilled track on some cool instruments:

Read more about this funky little instrument here.

Aftermath

Posted by on 2012/06/06 No comments

So we survived the day pretty well. Yay for things going as they should ;)

A quick summary would be having one query regarding being unable to hit our test site and that turned out to be a browser issue at the client. The following counters from it (stats from around 15h00 SAST):

   2012-06-06  --  228 IPv4 only
   2012-06-06  --  5 Confused
   2012-06-06  --  1 Web Filter
   2012-06-06  --  46 Dual Stack - IPv6 Preferred
   2012-06-06  --  16 Dual Stack - IPv4 Preferred

Not bad, considering we only took it live sometime last night. Some other people didn’t get by quite so well on v6 day though. Yahoo was one of them. When trying to go to ‘www.yahoo.com’, we get redirected to ‘za.yahoo.com’ with the following DNS records:

vandali % host za.yahoo.com
za.yahoo.com is an alias for fd-fp2.wg1.b.yahoo.com.
fd-fp2.wg1.b.yahoo.com is an alias for ds-fp2.wg1.b.yahoo.com.
ds-fp2.wg1.b.yahoo.com is an alias for ds-any-fp2.wa1.b.yahoo.com.
ds-any-fp2.wa1.b.yahoo.com has address 87.248.112.181
ds-any-fp2.wa1.b.yahoo.com has IPv6 address 2a00:1288:f00e:1fe::3001
ds-any-fp2.wa1.b.yahoo.com has IPv6 address 2a00:1288:f006:1fe::3000
ds-any-fp2.wa1.b.yahoo.com has IPv6 address 2a00:1288:f006:1fe::3001
ds-any-fp2.wa1.b.yahoo.com has IPv6 address 2a00:1288:f00e:1fe::3000

This then blows up at one of their Accelerators:
whoohoo

Worth a slight thought, since Yahoo actually appears to see use over much of Africa.

All said and done, a fairly good day. Didn’t notice any major blowouts elsewhere in the internet (although I should note I wasn’t tracking all news), and I look forward to some write-ups by the usual people (Renesys, HE, Evilrouters, etc) in the next few days. We appear to remain one of the most well-connected IPv6 ISPs in South Africa, and in a pretty good position overall.

6th of the 6th, launch ALL the things

Posted by on 2012/06/06 No comments

So, it’s World IPv6 Day^W^W^WIPv6 Launch Day (rebranding lulz). Go test your IPv6 at some appropriate site like here (hosted in South Africa) or here (somewhere else than South Africa, I didn’t check).

Google also had some conference announcement about the next stage in Maps. I wonder if it’s this thing I saw on Google Maps last night: Mapmaker. It appears to be 4sq meets Waze meets .. something. Let’s see at 9am PST.

Update: here’s a screenshot.

Second update: (fuck you WordPress editor) apparently Mapmaker’s been live since mid-April. Talk about a quiet launch.

Mapmaker Intro Screen

 

Hai, can I hav sum intergnats plox?

Posted by on 2012/05/31 No comments

Alternative post title: IPv6 all up in hurr

Some years ago, before the age of cheap international access on local ISPs arrived here, dual-homing (or n-homing, depending on how pimp you were) on your residential connection was quite the fashion among .za tech-heads. But not the fancy sort with BGP and decent best-route selection, just a really grubby sort: two accounts, one local (as in .za routing table) and one international. You can read up about the full setup over here on Stefano’s site.

Due to the nature of the split, there was some fun. Fun in the order which things might come up, fun in which session’s routing is ready first, fun in DNS server overwriting, that sort of thing. Of course, I mean fun tongue-in-cheek, since it was mostly an annoyance. Especially when ddclient picks the wrong PPP session (“the config says ppp1, why are you using ppp0?”), or doesn’t want to ignore its cachefile (forcing you to wrap it in another script and delete the cache yourself), or when your line flaps and all pppd instances go into this weird race condition where they suddenly all acquire the same IP, or ….. well, I guess you get the idea. It was painful.

Thankfully times have progressed, and now it’s possible to get IPv6. Hell, if you’re in the right place you can even get a static allocation of v6. Working for AS37105, this is of course one of the work perks, since we (the tech team on the v6 deployment) dogfood it ourselves to make sure we know that things are actually working. Things we usually note are the following:

  • explosions in HE.net’s v6 core – hey, it happens
  • client apps misbehaving – surprisingly, chrome on my desktop is one of these
  • “mixed” support – mikrotik, for instance. you can telnet/ssh it on v6, but not winbox to a v6 address (I don’t recall if I’ve tested whether it connects if a hostname resolves to v6 address..mental note)

Personally, the best part for me is not having to ever deal with broken dyndns anymore, or having to maintain lots of funky NATs, or having to tunnel home and route traffic via the tunnel. If I just quickly want to ssh to my desktop, it has a hostname in DNS and it works. If I quickly want to check up on my traffic stats or anything else, I browse to yariman (my gateway/home store). It’s great, and makes my life that much nicer.

All of this said, World IPv6 Day next week! Are you all ready for your few days of carnage as other shitty ISPs run around unprepared? Bring on the future!

One other thing, props to PH.Fat for another good track. The track alone is cool enough for me to share it, but then I saw that the album (available on their website) is creative commons, and that just wins a bit harder. Nicely done, guys :)

P.S. Fuck you, WordPress content editor, and your stupidity in paragraph designation flow after bulletpoints.

Routing for n00bs

Posted by on 2012/05/09 No comments

In lieu of solar flares, and unicorns, I propose a new protocol name to use when dealing with people who don’t understand routing and friends, people to whom any level of tech explanation would be white noise:

MGP. Stands for Magic Gateway Protocol. It knows just what to do at all times.

Todo: write RFC.