“Tip me that output quickly, would you?”

Jonathan recently wrote Codetip, a pastebin based on Twisted.Web with some neat client-side features and integrating server-side JS as well.

There’s a demo one running at http://pb.vuze.la/

Short of the setup doc in the readme, there’s some further work you get to do when installing this on something like Debian Squeeze. Here’s a quick rundown from my shell history:

 1237  aptitude -t wheezy install python-pip
 1242  aptitude -t wheezy install virtualenvwrapper
 1252  aptitude -t wheezy install python-virtualenv
 1257  aptitude -t wheezy install python-sqlite python-pysqlite2

All of those mostly because of versioning. Node.js was also installed previously (for Brewer.js), so npm was around as well, at the sid package version level.

It was set up in a separate user account, with a virtualenv to avoid crapping all over the system. For those who don’t know, a virtualenv is used to make a little jail for python things to get installed into. This way you can have a different local version of something to what’s installed globally (globally meaning system level). Turns out this makes life a bit more painful:

Drop this into .zshenv (or the appropriate equivalent for your shell):

# virtualenv
export WORKON_HOME=$HOME/.virtualenvs
export VIRTUALENVWRAPPER_VIRTUALENV_ARGS='--system-site-packages'
export VIRTUALENVWRAPPER_LOG_DIR=$WORKON_HOME
export PIP_VIRTUALENV_BASE=$WORKON_HOME
export PIP_RESPECT_VIRTUALENV=true

And then run:

git clone https://github.com/jonathanj/Codetip.git
mkvirtualenv Codetip
activate Codetip
pip install Twisted; pip install Epsilon; pip install Axiom; ln -s =nodejs $(VIRTUAL_ENV)/bin/node
npm install brewer
cd Codetip
./node_modules/brewer/bin/brake install
./node_modules/brewer/bin/brake all
twistd -no-web --notracebacks --class=Codetip.resource.RootResource

The virtualenvwrapper package provides ‘mkvirtualenv’ amongst other things, use `source /etc/bash_completion.d/virtualenvwrapper` to get the hooks. This shellscript also works in zsh.

And that’s it, a working Codetip instance.

Zenoss Device Listing

Nice for if you need some form of automated report in zenoss, here’s a code snippet to print out a list of devices per class, formatted in markdown markup

typehash = {}
for d in dmd.Devices.getSubDevices():
    if typehash.has_key(d.getDeviceClassPath()) == False:
        typehash[d.getDeviceClassPath()] = []
    typehash[d.getDeviceClassPath()].append(d.viewName())

for key in typehash:
    print """### %s\n""" % (key)
    for item in typehash[key]:
        print "*   %s\n" % (item)
    print ""

Run it in the zenoss console, or anything with a dmd connection. Tested on 2.5.x, but should be trivial to port forward if it breaks.

(PS: I could probably fix up that last bit to do something with .iteritems() instead…)

Post-jump, things everywhere die

Always a bit interesting to check out stats for things on the internet after a major event like Felix Baumgartner’s jump from space today. Here’s a graph for JINX, one of the two .za INXs.

Interestingly, it seems to have all come from only one of the two GGCs that are visible via that path.

Elsewhere we see similar drops, and also some of the GGCs dying (502’s were served up for a while). Here’s the CAR-IX stats:

During the stream, some people were handed off to Akamai edge nodes. I don’t have that much information on what was happening in all the various networks, though. Will be interesting to see that coming to light soon from the likes of Renesys and such.

Update: I see the first image from this page made it onto one of the .za news syndicators without credit. Nice of them.

Old tricks, new coins; same problems

In setting up some mirrors recently, I’ve come to learn that rsync’s algorithm by default doesn’t deal well with long-haul TCP going hand in hand with small files. I still need to set aside some time to find a nice set of optimization flags to tweak that up a bit more. And when I say “doesn’t deal well”, I mean in the region of “can tx about 1/3rd a single-session TCP speedtest can do over the same path”. Later’s worry, though.

So, the point of this point:

receiver$ nc -l 1234 | pv | tar xf
sender$ tar cf - moz | pv | nc receiver.domain.tld 1234

And there you have a minimal-CPU-usage streaming file delivery setup. Of course, this doesn’t deal with retransmits or anything else. This just gets the bulk over. After that you can run an rsync with big block lengths over all of it, and get any files fixed using that.

And the “old trick” part of this? As far as I know, this is pretty much how the whole tape drive thing used to work[0]. I’ve just added some internet in the middle.

[0] – I wouldn’t know for certain, before my time

Elegua

Public Service Announcement

Anyone who makes use of elegua, the transition of services on it is now complete and I’ve updated the main A and AAAA records to point at the new host.

If you have any issues, you know where to find me.

(That said, the original TTLs were like six gazillion years or something, so caches might flush later as they go. Query the upstream NS for the new record if you need it.)

.co.za domains considered harmful

If anyone ever wants to register a .co.za domain, it looks like you’ll have three options going forward (from the near future):

  1. run away screaming
  2. commit suicide
  3. pay someone else to do it
That’s if we skip over the other practices they have, like refusing to allow you to register a domain if the NS records don’t exist on some servers yet (think about the workflow some DNS hosters take, this might at times be a perfectly normal scenario), or the weird whois setup that still seems to be the default server for most whois clients in the world.

Alongside my froztbyte.net domain, I also have a froztbyte.co.za from before I had a credit card. It’s useful for some stuff. But wow, dealing with coza is a trip. First, they only recently made an EPP interface available, and a quick scan-over of it looks like you need to be a registered/accredited registrar to use it, weighing in at R5000 (presently that’s just below 500eur). No matter, it’s not like I’m going to go find an EPP implementation now to do this. So the antiquated *email* interface it is.

Wander over to their website, grab the update form for my domain, edit it with the new NS info, submit. Wait.

mail:/var/log# tail -n 500 exim4/mainlog | grep 1TEc0u-0003kD-QC
2012-09-20 10:21:36 1TEc0u-0003kD-QC <= jp@domainiwanttoupdate.co.za H=(vandali.neology.co.za) [2001:43e8:8:1::x:x:x:x] P=esmtp S=8582 T="test Thu, 20 Sep 2012 10:21:16 +0200" from <jp@domainiwanttoupdate.co.za> for coza-admin@co.za
2012-09-20 10:21:38 1TEc0u-0003kD-QC == coza-admin@co.za R=dnslookup T=remote_smtp defer (-44): SMTP error from remote mail server after RCPT TO:<coza-admin@co.za>: host mx2.coza.net.za [82.103.142.199]: 450 4.2.0 <mail.neology.co.za[41.73.33.140]>: Client host rejected: Greylisted, see http://postgrey.schweikert.ch/help/co.za.html
2012-09-20 10:22:25 1TEc0u-0003kD-QC == coza-admin@co.za routing defer (-51): retry time not reached
2012-09-20 10:29:29 1TEc0u-0003kD-QC == coza-admin@co.za R=dnslookup T=remote_smtp defer (-44): SMTP error from remote mail server after RCPT TO:<coza-admin@co.za>: host mx2.coza.net.za [82.103.142.199]: 450 4.2.0 <mail.neology.co.za[41.73.33.140]>: Client host rejected: Greylisted, see http://postgrey.schweikert.ch/help/co.za.html
2012-09-20 10:32:05 1TEc0u-0003kD-QC == coza-admin@co.za R=dnslookup T=remote_smtp defer (-44): SMTP error from remote mail server after RCPT TO:<coza-admin@co.za>: host mx2.coza.net.za [82.103.142.199]: 450 4.2.0 <mail.neology.co.za[41.73.33.140]>: Client host rejected: Greylisted, see http://postgrey.schweikert.ch/help/co.za.html
2012-09-20 10:32:25 1TEc0u-0003kD-QC == coza-admin@co.za routing defer (-51): retry time not reached
2012-09-20 10:34:01 1TEc0u-0003kD-QC == coza-admin@co.za R=dnslookup T=remote_smtp defer (-44): SMTP error from remote mail server after RCPT TO:<coza-admin@co.za>: host mx2.coza.net.za [82.103.142.199]: 450 4.2.0 <mail.neology.co.za[41.73.33.140]>: Client host rejected: Greylisted, see http://postgrey.schweikert.ch/help/co.za.html
So I end up actually phoning my domain registrar, in 2012, to find out how long I need to wait. “Up to 45 minutes”. A few exim queue flushes later, the mail went through. Now I should receive the mail that allows me to respond with the the auth cookie. Oh, wait, no:
COZA: ERROR: Invalid phone number format supplied for the registrant phone or fax numbers “froztbyte.co.za”.

I first have to have a validation failure, because the data THEY SUPPLIED doesn’t confirm to their validation schema. This is also not a new thing. They’ve had various schema updates over various points of the ccTLD lifetime, and it’s often just a case of “struggle with it until you get it working”.

Now, given, they seem to have acknowledged that they fail at life as a registrar, thus the new EPP setup and accredited registrars. But for crying out loud, make some reasonable interface for people who aren’t on that system yet. Maybe I’ll do the effort of finding a good registrar….or but I’ll just stop caring about .co.za domains forever and move my stuff elsewhere.

Fun things to come home to

*sigh*….so much for the idea of doing work on Coursera thing (I just signed up for today) tonight:

yariman# tail -n 100 syslog | grep ppp
Sep 10 16:44:41 yariman pppd[24971]: Plugin rp-pppoe.so loaded.
Sep 10 16:44:41 yariman pppd[24972]: pppd 2.4.5 started by root, uid 0
Sep 10 16:45:16 yariman pppd[24972]: Timeout waiting for PADO packets
Sep 10 16:45:16 yariman pppd[24972]: Unable to complete PPPoE Discovery
Sep 10 16:46:21 yariman pppd[24972]: Timeout waiting for PADO packets
Sep 10 16:46:21 yariman pppd[24972]: Unable to complete PPPoE Discovery
Sep 10 16:47:26 yariman pppd[24972]: Timeout waiting for PADO packets
Sep 10 16:47:26 yariman pppd[24972]: Unable to complete PPPoE Discovery
Sep 10 16:48:31 yariman pppd[24972]: Timeout waiting for PADO packets
Sep 10 16:48:31 yariman pppd[24972]: Unable to complete PPPoE Discovery
Sep 10 16:49:36 yariman pppd[24972]: Timeout waiting for PADO packets
Sep 10 16:49:36 yariman pppd[24972]: Unable to complete PPPoE Discovery
Sep 10 16:50:41 yariman pppd[24972]: Timeout waiting for PADO packets
Sep 10 16:50:41 yariman pppd[24972]: Unable to complete PPPoE Discovery
Sep 10 16:51:46 yariman pppd[24972]: Timeout waiting for PADO packets
Sep 10 16:51:46 yariman pppd[24972]: Unable to complete PPPoE Discovery
Sep 10 16:52:51 yariman pppd[24972]: Timeout waiting for PADO packets
Sep 10 16:52:51 yariman pppd[24972]: Unable to complete PPPoE Discovery
Sep 10 16:53:00 yariman pppd[24972]: Terminating on signal 15
Sep 10 16:53:00 yariman pppd[24972]: Exit.

Line sync’d where it always has, good signal vs noise, etc. DSLAM or something in the middle just missing. Now to wait and hope my ticket gets to a useful support person. It *sucks* not having access to the local loop.

And Justin Case™ you couldn’t guess it, that post title is a lie.

Everything is not “just a string”

During a quick conversation on unicode and punycode, I managed to find http://☁→❄→☃→☀→☺→☂→☹→✝.ws

Cute, and a sad reminder of how many people still fight this.

Should I buy some stock?

thoughts: I still have some stock account balance left..wonder if I should buy some..

In [1]: from random import choice
In [2]: choices = {'buy': 0, 'wait': 0}
In [3]: for i in range(0,5000):
    choices[choice(['buy','wait'])] += 1
   ...:
In [4]: choices
Out[4]: {'buy': 2518, 'wait': 2482}

Guess I’ll buy some stocks.

[ed's note: this method works equally well to decide which stocks you want to buy when you're lazy]

We need this for internetworking

This. No, seriously, we really do.