coding SIMPLOT

I think there are two reasons that I didn’t become a programmer or computer scientist despite some enthusiasm in that direction at one point. One reason is that I wasn’t particularly good at programming. The other is that, despite not being very good at it, I found it addictive. Once I had a problem and a goal in mind — a little machine of code I wanted to build within the bigger machine of the computer — it was difficult to let go until I was able to watch all its little gears and cogs whirling around. When I’m building one of these machines, time seems to vanish. Suddenly it’s 3 a.m., I have bags under my eyes, and I haven’t eaten for hours. Somehow that never happens to me when I’m trying to finish writing an article.

My recent attempt to recreate the logic and aesthetic of a computer simulation of animal movements written in the mid-1960s sucked me into one of those time-holes. At each step of the way, I realized there was some language or tool I needed to learn how to use or some new feature I had to add, and what was initially intended to be a modest experiment ended by chewing up an amount of hours that I am too embarrassed to estimate. I’ll just say that I didn’t get out much for a couple of weeks. If you want to take a look at the code that eventually resulted, it’s available for perusal via Github. The following is a partial reconstruction of how it was produced.

FORTRAN forever

Siniff AMP Figure 10The program I eventually wrote was based on FORTRAN code that had been printed as an appendix to ecologist Donald B. Siniff’s dissertation, “A Simulation Model of Animal Movement Data,” which he submitted to the University of Minnesota in 1967. This code was my first time encountering FORTRAN, and there were some complications, beginning with the challenge of learning to read a programming language that looked somewhat different from those I was (or at one point had been) familiar with — namely BASIC, C, Lisp, and Python.

In case you’ve never seen FORTRAN before, here’s a snippet from Siniff’s program:

407 IF(P.EQ.1.0)402.403
402 RN=RANF(-1) $ M=1
406 RN=RN-AMOVE(M) $ IF(RM)404,404,405
405 M=M+1 $ GO TO 406
404 NUM=M $ MIN=(NUM*30)+15 $ GO TO 23

If you know FORTRAN, it’s pretty straightforward — a few simple calculations, comparisons, and control of the program’s flow. But there was virtually no documentation or in-code commenting for Siniff’s program, and it took me a while to figure out precisely how each statement worked and what the overall logic was. I got some help figuring out the FORTRAN part from old Control Data Corporation programming manuals scanned as part of the bitsavers collection hosted at the University of Stuttgart.

GOTO hell

siniff-code-snippet-marked-upThe biggest challenge, however, was simply to understand code written by somebody else a long time ago for his or her own purposes. In this case, the challenge was exacerbated by the use of a tangled web of “go to” statements rather than subroutines, which was typical for the time but which makes it harder to see the program’s logic. After wrestling with this code, using a more recent language like Python felt a breath of fresh air. And to imagine that the FORTRAN code had to be punched onto cards before it could be run and tested!

I’ve scanned in two versions of the original code: a clean version just as it appears in the printed-out microfilm copy of Siniff’s dissertation that I received through Inter-Library Loan (thanks to the fantastic librarians at the Max Planck Institute for the History of Science), and a photocopy of the print-out that I marked up with a pen during a few successive commutes on Berlin’s S-Bahn.

This is probably the right time to mention that when I told Siniff that I was re-creating his program from 1967, he seemed both mystified and a little embarrassed. He assured me that the version of the program in the dissertation had serious flaws that he was only able to correct in a later version. Nothing I say here about the difficulty of understanding the code should be taken as a judgment of the quality of Siniff’s work; on the contrary, this little project is partly meant to be a (geeky and perhaps misguided) sign of respect for a pioneering piece of work.

Machine translation

data-program-data-punchcardsOnce I thought I had figured out Siniff’s code well enough to serve my purposes, I faced the challenge of learning another language with its own special quirks: Javascript. I wanted to use Javascript to re-create the simulation because I knew it would simplify the task of putting the simulation online and making it interactive, so that somebody other than me might possibly use it. Running the original FORTRAN would have made for a more authentic re-creation, but Javascript was built for making interactive websites, and it is built into all browsers.

Fortunately the basics of Javascript are pretty easy to learn, although the translation from FORTRAN to Javascript did create some problems. The latter, unlike the former, has no “goto” function — that is, no way of telling the program to jump to a particular line of code. To replicate the logic of Siniff’s code, I had to redesign the control flow of the program without altering the underlying simulation and plotting logic.

Another challenge was getting used to an event-driven programming style that was necessary for making the simulation into a web app — that is, a style where the program runs by “listening” for the user to do things like click a button or enter a new value into a text field, rather than following a pre-determined execution order. I also struggled to get the aesthetics of the graphical output right using the HTML5 Canvas element. In retrospect, the controls, event listeners and plotting were simple enough, but I made a lot of small errors along the way that compounded into major headaches.

Feature creep

simplot-options-jpgOnce I had the thing running and generating outputs that resembled the illustrations in Siniff’s dissertation, I kept seeing areas where I could add a new feature and getting sucked back into the code. Some of these updates were minor (and not particularly faithful to the original), like making it possible to change the colors of the plot; some of them actually made the simulation more interesting. The most significant of these was the ability to select different distributions for the angle at which the simulated “animal” would turn at each step. This was one of the major focuses of Siniff’s work, and to leave it out, I thought, would be seriously undermining the value of the re-enactment, such as it was.

But to put that key piece of flexibility into the program required jumping through a few hoops. First, it didn’t make sense to have the option of choosing different probability distributions if they couldn’t be visualized — it would be impossible to understand what they were doing or how they differed. So I spent some time exploring graphing libraries for inserting simple bar charts into web pages, eventually settling on one of the simplest, which in fact was intended to produce tiny inline “sparklines.”

I then had to figure out how to generate the probability distributions themselves. Siniff had chosen the circular normal or von Mises distribution, but Javascript didn’t have a standard math library that could generate it easily. Python does have a “random” library with the von Mises distribution, though, so I wrote a small Python program that could generate distributions using different parameters. Some of the distributions would make the “animal” tend to continue going forward; others would make it more likely to turn left or right, or reverse direction at every step. I then cut and pasted the outputs into my Javascript program and added a drop-down selection menu to the interface. Not pretty, but it worked.

Emerging from the cave

simplot-plotting-progress-jpgOnce you’ve been deeply immersed in building a program for a while, it becomes almost impossible to imagine how someone who hasn’t been similarly immersed will encounter it.

After showing an early draft of the program to a friend, I realized that even though it was completely obvious to me what was going on when a simulation was plotted, it wasn’t going to be obvious to someone coming to the program without prior knowledge. I had set up the program to draw the whole simulation instantaneously, so it was hard to see that it was actually a path that was being traced out. It just looked like a static pattern — there was no movement in the animal movement simulation.

This was faithful to the original, since Siniff had received his results as finished plots, but it undermined the impact of the re-enactment. Adding the option of plotting at different speeds, with a progress bar showing how far through the simulation the plot had progressed, wasn’t quite as difficult as adding a choice of probability distributions, but it also took some time. I think I still prefer seeing the patterns plotted without any delay, but the fact that you can see it plotted out step-by-step is crucial.

Finally, I spent a while tweaking the web interface, trying to make it as easy to use as possible while also providing all of the options I had painstakingly added and maintaining some of the minimalist aesthetic that had attracted me to the idea in the first place. And then I clicked “run simulation” a few (hundred) times …