Category: day-job

I've just finished my first week in a new job, which is looking like it's going to be a lot of fun. I'm working for MemCachier, a small company (I'm employee #4) that offers a multi-tenant cloud-based memcache service. The code is mostly Haskell and Go (and maybe in the future…

In particular, getting from where you are now to where you want to be, in terms of your career.

As a result of an email I sent to the Haskell-Cafe mailing list a couple of weeks ago looking for someone to take over a contract I had been working on, someone contacted me asking for career advice. Clearly not someone who knew me at all, otherwise they would have known what a crazy idea that was. Anyway, this person was asking about one of the fundamental problems when you're starting out in more or less any profession: how do you acquire the experience you need to apply for jobs that say "experience required", which is more or less all of them?

They asked: "What is the path to getting involved in this stuff? How do I bridge the gap from just playing around with these technologies to having real world experience? It seems that most opportunities are for people with experience." And this is exactly right. Particularly for contracting, no-one wants to hire someone they think will have to learn on the job. You need to know what you're doing, which means getting experience somehow. And it would of course be nice to be able to eat and have a life while getting that experience.

I wrote an epic email in reply, and was told that it would have worked better as a blog post (or perhaps a short novel). So here I am, turning it into a blog post!

For BayesHive, one of the things we need to do a lot of is program transformation. The main reason for this is to generate code to represent probability distribution functions derived from descriptions of probability models -- the models are described using code written in the probability monad, and this needs to be manipulated quite extensively to determine the PDF, which is needed for the Markov chain Monte Carlo sampling we use for estimating parameters in probability models.

Program transformation is something that Haskell is really great at, since it basically just involves pattern matching against lots of different cases of syntax trees in the language you're transforming. Unfortunately, this leads to lots of code that looks like this (picking a simple case at random):

...
case last rvs of
   Observed (EApp (EApp (EVar "map") (EVar f)) (EVar nm)) -> rewriteFst f nm
   _     -> rvs
...

All that stuff with EApp and EVar is the explicit AST representation of the expression map f nm in the language (Baysig) that we're manipulating.

At first, doing things like this is pretty unavoidable, but wouldn't it be nice if you could write something like this instead?

...
case last rvs of
   Observed [baysig|map  nm|] -> rewriteFst f nm
   _     -> rvs
...

For the past few months, I've been spending most of my working hours on BayesHive, the brainchild of Tom Nielsen, CEO of OpenBrain. This is a web platform for doing Bayesian data analysis, intended to make these powerful methods more accessible for scientists and other data analysts.

We opened the system to some alpha users a while ago, but this week we've decided we're ready for wider beta testing, for people to have a look, kick the tyres and tell us what they think.

I've mostly been working on the web app part of the system, but in this post I want to talk a little bit about the general idea behind the system and the Bayesian approach to statistics. (As far as BayesHive goes, the system is made up of a reasonably complicated single page web app using the AngularJS framework, a Yesod server process to deal with the management of documents, data and models, and a model compiler and inference engine that does all the clever stuff.)

In the last three days, I've averaged 10 hours of billable time per day. That's ten hours per day actually standing at my desk coding, perhaps 10-20% more than that "on the job". I've been working on social and search aspects of the BayesHive web app: we have a pile of different…

Ah, another involuntary blogging hiatus. This time, for a good reason though. I have work. And it's work that pays money, is interesting and has some future in it. It's pretty cool. I've also had an offer of more work that I've had to defer until next year, which is encouraging.…

Some recent-ish papers of mine, that is. Although I didn't enjoy my last post-doc in Montpellier a whole lot, there was a small amount of output from it, which probably deserves recording. So here goes.

So, over the weekend Rita and myself made a big step. We both sent in resignation letters for our jobs in France! Neither of us likes living in Montpellier and, although Rita's job is more interesting for her than mine is for me, she wants to have a change of career and is trying…

I was up at our experimental site at Puéchabon the other day, installing a new PC to collect images from our webcam. The previous one had been stolen by some local yokels, along with all the solar panels and the electronics to go with them, and a load of other hardware. The…

One of the things I'm responsible for in my day job is a phenology webcam at our experimental site at Puéchabon. The idea of this is to observe colour changes in the canopy of the forest up there, with a view perhaps eventually to replacing manual phenological observations with information drawn from digital photos. In this sense, phenology means things like when flowers come out, when fruit forms, and so on. The Quercus ilex (holm oak) forest at Puéchabon is evergreen, so you don't get the same spectacular seasonal changes in leaf colour that you see in deciduous forests, so we're not sure whether this is going to work -- the changes we'll be looking at will be a bit more subtle.

by Roger S. Bivand, Edzer J. Pebesma & Virgilio Gómez-Rubio

I recently had to do a bunch of geostatistical analysis on some climate data (to be specific, using universal kriging to interpolate a time series of solar radiation data covering the region I'm working on to a different grid). I started off trying to use the geostatistical analysis toolbox in ArcGIS, which works fine as far as it goes, but seems to be very difficult indeed to access via ArcGIS's Python scripting interface. Since I had 36 years of daily data to process, doing it by hand was not an option.

In my day job, I work on the ecology of Mediterranean ecosystems in southern France, and if you work on a particular type of ecosystem, the first thing you need to know is where you can find your ecosystem! Seems like a simple problem, but it's not.