Category: mathematics

I was out walking Winnie the other morning and got to thinking about the storage capacity of neural systems (I'm reading Nils Nilsson's The Quest for Artificial Intelligence at the moment, which was what triggered this line of thought). I'll write about that in another article, but I realised as I was figuring through this that I would end up talking about some rather large numbers. Everyone throws around big numbers (millions, billions, trillions, and up and up), but how often do you stop to visualise what these things mean?

There are some nice animations around that help with thinking about relative scales (for instance, this one is really good). What these things don't do is to give you a sense of numbers. What does a billion of something look like? Can you get a really strong physical feeling for numbers of this magnitude and larger?

Here, I'll show you what I do. It's nothing ground-breaking, and it's kind of obvious once you get started, but it is effective. It's also quite a nice mental exercise.

This won't sound like fun (or Fay!) to start with, but we'll get there...

In the previous article, we looked at some of the details of a scheme for calculating the Lyapunov exponents of a system of differential equations. In this post, we'll see how to implement this scheme in Haskell. The code we describe is available as a Gist here.

The archetypal chaotic dynamical system is the Lorentz system, defined by the differential equation system

where , and are real constants. On the face of it, this looks like a fairly innocuous set of differential equations. However, as is well known, for a wide range of choices for the parameters , and , this system of equations has chaotic orbits, with sensitive dependence on initial conditions and an attractor of fractal dimension.

Determining whether a given dynamical system will exhibit chaotic behaviour is, in general, difficult . The orbit structure of chaotic systems can be very complex, and it can be difficult to distinguish true chaos from a number of related phenomena.

One approach to characterising the orbit structure of dynamical systems is via Lyapunov exponents, a spectrum of values that measure the stretching and squashing of orbits -- "normal" chaotic systems normally have at least one positive Lyapunov exponent and are dissipative, which means that the sum of all of their Lyapunov exponents is negative. While it is relatively straightforward to define Lyapunov exponents, in practice calculating them is tricky. We'll see how to do this in some practical cases in this and the next couple of articles, following mostly the approach of Rangarajan et al. (1998).

The motivation for this series of articles was basically that I have recently been wondering about how practical it would be to use Haskell for more "traditional" mathematics: there is a big body of work and code relating to category theory and type theory in Haskell, but I've not seen so much about matrix algebra, differential equations, dynamical systems and so on. I implemented the Rangarajan method in Mathematica some years ago and thought it might be a good test case. I'm going to gloss over a lot of technical details about the computation of Lyapunov exponents since my main interest is in the Haskell implementation issues.

In 1998, the mathematical physicist David Ruelle wrote an odd little article, entitled Conversations on Mathematics with a Visitor from Outer Space, which appeared as a chapter of the very interesting looking book Mathematics: Frontiers and Perspectives, published by the IMU in 2000. I've not read the whole volume, although I think I've read at least one of the other chapters as a preprint. Most of the 30 authors seem to have written more or less "straight" articles, but Ruelle's is different. He wanted to explore the constraints imposed on human mathematics by the structure and capabilities of the human brain. Perhaps constraints is too strong a word -- "predispositions" might be better: human mathematics is necessarily tied to human brains, and human brains have evolved to their current state to solve problems that are quite different from the problems encountered in mathematics. One might thus expect human mathematics to have followed the "fault lines" in the Platonic edifice of Mathematics that are most easily appreciated using the mental tools that evolution has given us.

To explore this idea, Ruelle introduces the conceit of a visitor from outer space, a "galatic mathematician" pursuing doctoral studies investigating the nature of human mathematics. She describes some of the characteristics of the human mental apparatus that are relevant: limited short-term memory, and hence inability directly to execute complex algorithms; predisposition to see symmetry and pattern; human brains operate very slowly, even compared to human computers; the importance of geometrical and visual reasoning; and so on. Ruelle comments that despite our limitations, humans have managed to do some fairly complex mathematics, but the question is always: how much further could we go if we could transcend the limitations of our history?

Pathetically slow in starting work on my little constraints project as I am, here's the first of what should be a long series of posts...

One of the constraint solvers I'm looking at starting from, Cassowary, uses what is essentially an extended version of one of the most venerable of optimisation algorithms, Dantzig's simplex algorithm. I'm going to start off thinking a little bit about the general setup of the kind of linear programming problems that the simplex algorithm is designed to solve, just to get a geometrical feeling for how these algorithms work and to understand the issues that might arise from relaxing some of the assumptions used in them.

by Mark Kac & Stanislaw Ulam

This interesting little book, published in 1968, is the result of a collaboration between two of the great figures of 20th century mathematics. Although known mostly for his work on the Manhattan Project and later associated applied mathematics efforts (I first learnt of his work after being told to read about the Fermi-Pasta-Ulam problem by my PhD supervisor), Ulam began his career as a pure mathematician, working on problems in general topology. (His autobiography, Adventures of a Mathematician, is well worth a read.) Kac, best known for asking "Can one hear the shape of a drum?", made major contributions to probability theory. Kac and Ulam got to know each other in Poland, then both made the move to the United States during the Second World War.

Week 3 of the Stanford AI class was about machine learning. This was kind of handy for me, since my PhD thesis was mostly about applying some unsupervised learning techniques to dimensionality reduction for climate model output. That mean that I've read hundreds of papers about this stuff so the basic ideas and even quite a few of the details are already pretty familiar. However, most of what I've read has been aimed at applications in climate science and dynamical systems theory. Not much from the huge literature on clustering methods, for instance.

Sebastian very briefly mentioned some of the nonlinear dimensionality reduction methods that have been developed for unsupervised learning applications, but it was nothing more than a mention (no time for anything else). I can't resist the temptation to dust off a little of this stuff from the archives.