Category: AI

I recently read The Quest for Artificial Intelligence by Nils Nilsson. Interrupting a fairly linear history of the AI field is an interlude on some more philosophical questions about artificial intelligence, minds and thought. Searle's Chinese Room, things like that.

I got to thinking about some of these questions on my daily peregrinations with Winnie, and I started wondering about the scales that are involved in most discussions of minds and AI. Searle talks about an individual person in his Chinese Room, which makes the idea of some sort of disembodied intelligence actually understanding the Chinese sentences it's responding to seem pretty absurd. But is that in any sense a realistic representation of a brain or a human mind?

In keeping with my upbringing as a baby physicist, I'm going to take a very reductionist approach to this question. I want to get some idea of exactly what the information storage capacity of a human brain is. I'm deliberately not going to think about information processing speeds (because that would involve too much thinking about switching rates, communication between different parts of brains, and other biological things about which I know very little). I'll treat this in the spirit of a Fermi problem, which means I'll be horrifyingly slapdash with anything other than powers of ten. There will be big numbers.

by Nils Nilsson

Nils Nilsson has been involved in artificial intelligence research more or less since the inception of the field in the late 1950s. (He's probably best known for his work on Shakey the robot, A* search, and his idea of teleo-reactive planning.) This makes him ideally placed to write a history of artificial intelligence.

I recently read an interesting article by John McCarthy that set me thinking. The paper is called The Well Designed Child and talks a bit about about the old nature versus nurture debate: to what extent is a newborn child a blank slate, and to what extent are human intellectual capabilities intrinsic and instinctual?

To me, the whole question of nature versus nurture has always seemed odd. It's clear that both aspects are important. We are all limited to one extent or another in what we can do purely by the physical parameters of our existence: as much as I would like to teleport myself to Europa to frolic in the sunless sea beneath the ice, I can't, and never will be able to. Equally though, the potential that exists in all of us at birth can be squandered, and without education and opportunity, none of us can make of ourselves all that we might. So, we need a bit of both.

One thing that came to an end over the Christmas holidays was the Stanford Artificial Intelligence course. The final exam was on the weekend of 17/18 December and final grades were distributed a couple of days later.

Overall, it was an interesting experience, and I certainly learnt some things. The instructors (Peter Norvig and Sebastian Thrun) were enthusiastic and clearly extremely knowledgeable, and they took on an extremely challenging task in trying to run an online class for tens of thousands of students. For that, I'm very grateful. There are a few things that weren't 100% ideal about the presentation of the class material, but that's to be expected the first time out with something like this.

The highlight of the course was definitely Sebastian's presentation of his work on self-driving cars. I'd heard a bit about the DARPA Grand Challenge before, but I hadn't known about the progress that the Stanford team had made with self-driving cars in traffic. That's really impressive work, and Sebastian did a nice job of relating the algorithms used for guiding cars to the simplified versions considered in the class.

Week four of the Stanford Introduction to AI class was about, well, what was it about? It was a bit of a mish-mash. Some stuff about logic, some stuff about planning. Nothing I can really pin some experiments on, so I've been thinking a bit about the format of the course instead.

Week 3 of the Stanford AI class was about machine learning. This was kind of handy for me, since my PhD thesis was mostly about applying some unsupervised learning techniques to dimensionality reduction for climate model output. That mean that I've read hundreds of papers about this stuff so the basic ideas and even quite a few of the details are already pretty familiar. However, most of what I've read has been aimed at applications in climate science and dynamical systems theory. Not much from the huge literature on clustering methods, for instance.

Sebastian very briefly mentioned some of the nonlinear dimensionality reduction methods that have been developed for unsupervised learning applications, but it was nothing more than a mention (no time for anything else). I can't resist the temptation to dust off a little of this stuff from the archives.

The second week of the online Stanford AI class was about probability, reasoning when there is uncertainty and, more specifically Bayes networks. I don't much want to talk about the theory behind these things here since that's covered in a lot of detail on the linked Wikipedia page, in the lectures and in AIMA.

Instead, I want to present a mildly amusing, although not very efficient, Haskell implementation of Bayes networks. It's often said that if you know the full joint probability distribution function (PDF) for a system, you can calculate any marginal or conditional probabilities you want. Well, Haskell is a functional programming language, a PDF is a function, so can we represent the PDF explicitly and use it for calculating? The answer, of course, is yes.

In an earlier post, I presented some code for solving tile puzzles using A* search. In the lectures, Peter Norvig talked about two different heuristics for these puzzles, one simply counting the number of misplaced tiles and the other giving the total Manhattan distance of all misplaced tiles to their correct positions. It's interesting to find out how much of a difference using these heuristics really makes.

The first week of the online Stanford AI class has gone by. The format of the presentation is pretty good, with lots of short lecture videos chained together with little quizzes embedded. It's really pretty neat. There are some minor errors in some of the lectures, which is normal, though in this kind of online setting, it's more difficult for the keener students to correct the lecturer as the lecture goes along, so some people are likely to spend a bit of time confused because of that. Some people have complained on Reddit about the relatively superficial level of the coverage of the lectures, but as far as I'm concerned, that's what the book (and papers) are for. The course is called "Introduction to Artificial Intelligence", after all.

The two things we covered this week were a very quick introduction to the field, then some stuff about search. Although there is a big pile of code available for download associated with Russell & Norvig's AI book which would have made playing with some of this stuff easier, I decided to write some code of my own in Haskell, mostly for my own amusement. I wanted to implement A* search, with a clean interface for setting up problems.

Along with about 160,000 other people, I've signed up for the Stanford Introduction to Artificial Intelligence course. My interest in this is two-fold: first for the material, which should be pretty cool if Peter Norvig's book is anything to go by, but second for just how they…