Remote ipython notebooks

I use ipython notebooks a lot. For interactive tinkering, for calculations with units, for interactive control of long-running processes, for easy parallelization (details to come), and to generate plots for all my papers. When I can, I run the notebooks on a server rather than my rather wimpy laptop; apart from the obvious advantages of more cores, more disk space, and more RAM, this also means I can leave things running for ages. It's all very convenient, but it requires some setup. As with the ssh configuration, I've found myself explaining the setup to several different people recently, so I thought I'd put it here. (Still working on that cute-kittens post.)

SSH incantations

I use ssh all the time, for shell access on remote machines, to transfer files, to carry VNC sessions, and to use different institutional affiliations to get at paywalled things. Over the years I have developed a particular way of setting it up that minimizes the pain involved in doing all of the above things. Several times in the past months I have tried to explain the whole setup to someone, so I think that's a sign it's time to post it here. I'll try to post something about, I don't know, fluffy kittens, boating on the Thames, seaborgium hexacarbonyl, or something soon for those of you who for some reason expected an interesting blog.

git tear-hair

I use git. It's fast, it's convenient, it gives you github, and mostly it works. But several times now I've managed to bollix a git repository, been faced with impenetrable messages, and been unable to continue using git unless I sort them out. This latest time I had enough stubbornness to figure out how to un-bollix my repository, and I wanted to record it for posterity (specifically, for people whose problem-solving technique involves Googling for the specific error message). The error I received was:

error: object file .git/objects/86/5d2dffe9a3d72917934ed9693c7167efb6d8d5 is empty
fatal: loose object 865d2dffe9a3d72917934ed9693c7167efb6d8d5 (stored in .git/objects/86/5d2dffe9a3d72917934ed9693c7167efb6d8d5) is corrupt

Read on for how it happened, my understanding of what it means, and how I fixed it. Or skip this even-more-technical-than-usual post; I'll try to post something with corals or kittens soon.

Old-school UNIX nerdery

I've never owned a Windows machine, or a Mac. It's not that I'm particularly hardcore about open source, it's just that I got used to a command line back in the old VIC-20/DOS/DesqView days and never really got away from it. Our undergrad computer network was very convenient, but made up of about four flavours of UNIX, and in fact they had a load-distributing program at login that meant your initial login was not just to a random machine but to a random UNIX flavour. Somehow the IT folks made this work tolerably well. After a few years of that, when I finally got a home computer, I chose to run Linux on it.

ssh shenanigans

I use ssh all the time, from simple terminal connections to VNC sessions controlling telescopes to rsyncing terabytes of data to serving my music and video library. Most of the time, the machine I actually want to connect to is not visible from the open internet, and in some cases there are two layers of machines to go through to get to the one I want. I have used various ad-hoc combinations of ssh connections, but I just found a brilliant library that solves several of my problems: paramiko.

(Also, the name is from Esperanto, and the releases are named after gashlycrumb tinies. This is some serious nerdery, here.)

Rest in peace, Dennis Ritchie

Dennis Ritchie, who created the C language, has died. It would be difficult to overstate the importance of his work — C is imperfect and limited, but it serves as a near-universal tool for programming everything from tiny ATTiny8s to Blue Gene supercomputers. It does exactly what it was designed to do — provide a portable abstraction that is nevertheless close enough to the underlying machine to be efficient, and comfortable enough to write substantial programs in. These days, I (and many other programmers) prefer to write in Python or another high-level language, but most interpreted languages are still implemented in C, and when performance is critical, C is generally the way to go. Dennis Ritchie's impact on the world of computers was tremendous, and he will be sorely missed.

From: Dennis Ritchie 
Newsgroups: comp.std.c
Subject: Re: Computing sizeof() during compilation
Organization: Bell Labs, Lucent Technologies

> You are right.  It was nice back in the days when things like
> 
> #if (sizeof(int) == 8)
> 
> actually worked (on some compilers).

Must have been before my time.

 Dennis

bash pipes

I've been using UNIX for a very long time, and bash has been my shell for almost all that time, but for the life of me I can never remember how to pipe standard error anywhere. I think my problem is I've never found any logic by which the syntax makes sense. Anyway, it's:

command 2>&1 | less

But if you want to send the standard error and output to a file, it's the other way around:

command > file 2>&1