It's hard to believe that I've not written anything here since 2017. That's a long break. But now I'm back, with lots of new ideas and projects and stuff!

Anyway, what's happened in the last couple of years? Quite a lot. Here are the highlights:

• Houses sold: 1
• Jobs quit: 2
• Passports surrendered: 1

Last spring, I was taking a look at what I'd been reading recently. I read a lot of novels. Reading a novel seems to be my default state. I read a lot of science fiction, some crime novels, some historical fiction, some "straight" fiction (whatever that really is). But what I noticed was that the authors I tend to read skew very male. I decided I needed to do something about that, if only because I was probably missing some great stuff. So for six months I decided to read only novels written by women.

And I was right! I was missing some great stuff. I read about 60 novels in those six months, and discovered some amazing things that I really wish I'd known about before. Here are a few of the stand-outs.

I had to do a slightly weird bit of system admin recently, and there was one step that was kind of sneaky and not something I'd seen mentioned anywhere else. So I thought I'd better write it up...

What I wanted to do was make my desktop machine dual boot Arch Linux and Windows 10 (I needed Windows to use some CAD software). That's not so unusual, but the recommended procedure seems to be to start from a clean machine, install Windows, then install Linux. I was starting from a pre-existing Arch Linux installation and didn't want to lose anything. I also had things set up using syslinux in BIOS mode, which wasn't going to work with Windows. So I needed to switch to UEFI booting first.

I've just finished my first week in a new job, which is looking like it's going to be a lot of fun. I'm working for MemCachier, a small company (I'm employee #4) that offers a multi-tenant cloud-based memcache service. The code is mostly Haskell and Go (and maybe in the future…

Well, that's pretty poor. It's a year and a half since I last wrote an article here. A lot has happened in that time, and I have some cool new things coming up that I want to write about, so I'd better dust this off. Quick summary of the last 18 months: I started a new job in…

I recently did some work for Andy Ridgwell, an old colleague from Bristol, writing a build and configuration system and GUI for a medium-sized climate model called GENIE. GENIE is an EMIC, an Earth system Model of Intermediate Complexity. It's about 55,000 lines of Fortran and includes models of the atmosphere and ocean plus models of atmospheric chemistry and biogeochemistry in the ocean and ocean sediments.

This model had been in use for some years by different groups, and the infrastructure around it had become quite baroque. Andy wanted this tidied up and made nice (i.e. rewritten...) to make the model easier to set up and use. He also wanted a cross-platform GUI for configuring and running the model, allowing you to keep track of the model state in real-time, to pause and restart model runs, changing the model configuration in between, and so on.

A major consideration for this work was that as well as being easy to use the new system had to be easy to install (on both Linux and Windows) and easy for scientists to hack on. That ruled out Haskell, my usual tool of choice. I decided to use Python instead, for a couple of reasons.

One of the things that C2HS is lacking is a good tutorial. So I'm going to write one (or try to, anyway).

I've started doing a new thing this year to try to help with "getting things done". I normally have a daily to-do list and a list of weekly goals from which I derive my daily tasks, but I've also now started having a list of quarterly goals to add another layer of structure. Three months is a good timespan for medium-term planning, and it's very handy to have that list of quarterly goals in front of you (I printed it out and stuck it to the front of my computer so it's there whenever I'm working). Whatever you're doing, you can think "Is this contributing to fulfilling one of my goals?" and if the answer is "No, watching funny cat videos is not among my goals for this quarter", it can be a bit of a boost to get you back to work.

So, how did I do? Not all that badly, although there were a couple of things that fell by the wayside.

OK, so we're done with this epic of climate data analysis. I've prepared an index of the articles in this series, on the off chance that it might be useful for someone. The goal of this exercise was mostly to try doing some "basic" climate data analysis tasks in Haskell, things…

This is going to be the last substantive post of this series (which is probably as much of a relief to you as it is to me...). In this article, we're going to look at phase space partitioning for our dimension-reduced PCA data and we're going to calculate Markov transition matrices for our partitions to try to pick out consistent non-diffusive transitions in atmospheric flow regimes.

I took over the day-to-day support for C2HS about 18 months ago and have now finally cleaned up all the issues on the GitHub issue tracker. It took a lot longer than I was expecting, mostly due to pesky "real work" getting in the way. Now seems like a good time to announce the 0.25.1 "Snowmelt" release of C2HS and to summarise some of the more interesting new C2HS features.

This is going to be the oldest of old hat for the cool Haskell kids who invent existential higher-kinded polymorphic whatsits before breakfast, but it amused me, and it's the first time I've used some of these more interesting language extensions for something "real".

(There's no code in this post, just some examples to explain what we're going to do next.)

Suppose we define the state of the system whose evolution we want to study by a probability vector -- at any moment in time, we have a probability distribution over a finite partition of the state space of the system (so that if we partition the state space into components, then ). Evolution of the system as a Markov chain is then defined by the evolution rule

where is a Markov matrix. This approach to modelling the evolution of probability densities has the benefit both of being simple to understand and to implement (in terms of estimating the matrix from data) and, as we'll see, of allowing us to distinguish between random "diffusive" evolution and conservative "non-diffusive" dynamics.

We'll see how this works by examining a very simple example.

The analysis of preferred flow regimes in the previous article is all very well, and in its way quite illuminating, but it was an entirely static analysis -- we didn't make any use of the fact that the original data we used was a time series, so we couldn't gain any information about transitions between different states of atmospheric flow. We'll attempt to remedy that situation now.

What sort of approach can we use to look at the dynamics of changes in patterns of ? Our parameterisation of flow patterns seems like a good start, but we need some way to model transitions between different flow states, i.e. between different points on the sphere. Each of our original
maps corresponds to a point on this sphere, so we might hope that we can some up with a way of looking at trajectories of points in space that will give us some insight into the dynamics of atmospheric flow.

A quick post today to round off the "static" part of our atmospheric flow analysis.

Now that we've satisfied ourselves that the bumps in the spherical PDF in article 8 of this series are significant (in the narrowly defined sense of the word "significant" that we've discussed), we might ask what to sort of atmospheric flow regimes these bumps correspond. Since each point on our unit sphere is really a point in the three-dimensional space spanned by the first three PCA eigenpatterns that we calculated earlier, we can construct composite maps to look at the spatial patterns of flow for each bump just by combining the first three PCA eigenpatterns in proportions given by the " " coordinates of points on the unit sphere.