El Radar

I had the opportunity to visit the great radio telescope at Arecibo ("El Radar") last year. It's an astonishing machine, built more like a stadium than a telescope. Sadly, my digital camera died on that trip, so I don't have much in the way of pictures, but I came across this neat youtube clip (above). It's a segment from a BBC program, talking about dark matter and the role of Arecibo in understanding it. I thought it made a nice view of the telescope. I especially like that in some of the audio you can hear the coquì, little tree frogs that provide a constant chorus. (Of course the telescope is also in Contact and some James Bond film — surprisingly the latter provides a better view of the actual telescope.)

I'll talk a bit about the telescope and my own visit, with a few photos, below the jump.

The People's Microphone

I went down to Occupy Montreal the other day. It was pretty impressive, really; the modest park, surrounded by banks, was filled with tents and people making art. Particularly amusing was the statue of Queen Victoria wearing a Guy Fawkes mask (pictured on the right). But what got me thinking was the People's Microphone.

ssh shenanigans

I use ssh all the time, from simple terminal connections to VNC sessions controlling telescopes to rsyncing terabytes of data to serving my music and video library. Most of the time, the machine I actually want to connect to is not visible from the open internet, and in some cases there are two layers of machines to go through to get to the one I want. I have used various ad-hoc combinations of ssh connections, but I just found a brilliant library that solves several of my problems: paramiko.

(Also, the name is from Esperanto, and the releases are named after gashlycrumb tinies. This is some serious nerdery, here.)

Local Maxima and social annealing

Dan's Data has a new post/magazine article up, about local maxima. Frustratingly, there's no comment section, but I've been reading some interesting things that seem relevant, so I'll post them here. (TL;DR of Dan's post: socially we seem to get stuck in local maxima instead of finding the real best way to do things.)

Rest in peace, Dennis Ritchie

Dennis Ritchie, who created the C language, has died. It would be difficult to overstate the importance of his work — C is imperfect and limited, but it serves as a near-universal tool for programming everything from tiny ATTiny8s to Blue Gene supercomputers. It does exactly what it was designed to do — provide a portable abstraction that is nevertheless close enough to the underlying machine to be efficient, and comfortable enough to write substantial programs in. These days, I (and many other programmers) prefer to write in Python or another high-level language, but most interpreted languages are still implemented in C, and when performance is critical, C is generally the way to go. Dennis Ritchie's impact on the world of computers was tremendous, and he will be sorely missed.

From: Dennis Ritchie 
Newsgroups: comp.std.c
Subject: Re: Computing sizeof() during compilation
Organization: Bell Labs, Lucent Technologies

> You are right.  It was nice back in the days when things like
> 
> #if (sizeof(int) == 8)
> 
> actually worked (on some compilers).

Must have been before my time.

 Dennis

Curve fitting part 5: PyMC

I previously talked about fitting a curve to data — specifically, a sinusoid-plus-constant to a distribution of photon arrival times. I wrote some special-purpose Bayesian code to evaluate the relevant integrals directly so that I could work out the posterior distribution of fitted parameters. As a bonus I was also able to test the hypothesis that the photons were pulsed. Unfortunately, the method I used really doesn't generalize to more complicated situations. Fortunately, there is a toolkit that does: PyMC.
PyMC provides tools to implement Markov Chain Monte Carlo algorithms. These evaluate probability integrals numerically by generating samples from the distribution. They do this fairly efficiently by "jumping" semi-randomly from point to point in the space, but jumping preferentially towards points that are more likely. The algorithm is carefully arranged so that, after a certain number of steps, the initial guess is forgotten and the points are correctly distributed. They are correlated, but the correlations can be reduced or eliminated by subsampling the sequence. It's complex, finicky code. Fortunately I don't need to write it, because the algorithm is sufficiently generic that with PyMC all I have to do is specify the model I'm trying to fit. And, sure enough, it works. Code and results below the jump. The short version is: PyMC is easy to use and well-documented.
The second half of my previous post, model selection, turns out to be a little trickier to implement with Markov Chain Monte Carlo; I'll address that in a later post.

Testing whether a signal is broad-band

Radio pulsars are generally broad-band sources — you can hear them over a very wide range of radio frequencies, from about a hundred megahertz up to, in a few cases, tens of gigahertz. Their emission does change with frequency, decreasing as a power law, but over reasonable bandwidths we expect to see their signal in all frequency channels.


Radio-frequency interference ("RFI"), on the other hand, is very frequently narrow-band, appearing in just one, or a few frequency channels. In fact, one way we try to manage RFI is by using really wide bandwidths, so that there's so much power from the pulsar signal that narrow-band RFI is drowned out. Unfortunately, all too often the RFI is so strong that even a narrow-band signal can dominate the total power. And since it's narrow-band, dedispersion doesn't smear it out like it would a broad-band interference spike. So narrow-band RFI is one of the kinds of interference that is particularly hard to sift out from pulsar candidates.

Just recently, on the arxiv, a paper came out that attempts to address the problem of testing whether a candidate is broad-band:

Multimoment Radio Transient Detection
Laura Spitler, Jim Cordes, Shami Chatterjee, Julia Stone
We present a multimoment technique for signal classification and apply it to the detection of fast radio transients in incoherently dedispersed data. Specifically, we define a spectral modulation index in terms of the fractional variation in intensity across a spectrum. A signal whose intensity is distributed evenly across the entire band has a much lower modulation index than a spectrum with the same intensity localized in a single channel. We are interested in broadband pulses and use the modulation index to excise narrowband radio frequency interference (RFI) by applying a modulation index threshold above which candidate events are removed. The technique is tested both with simulations and using data from sources of known radio pulses (RRAT J1928+15 and giant pulses from the Crab pulsar). We find that our technique is effective at eliminating not only narrowband RFI but also spurious signals from bright, real pulses that are dedispersed at incorrect dispersion measures. The method is generalized to coherent dedispersion, image cubes, and astrophysical narrowband signals that are steady in time. We suggest that the modulation index, along with other statistics using higher-order moments, should be incorporated into signal detection pipelines to characterize and classify signals.

This looks very promising, but there's some testing I wish they'd done. More on the subject below the jump.