Team Human vs. Team Watson Round III

Well it seems the machines have won, select your pod early, you’ll want to get a good view of the energy harvesting machines.

We were talking about Watson around the ol’ AI research lab today and someone pointed out that Watson is yet another highly tailored solution to a particular problem just like DeepBlue (click it, its Arcade Fire, just CLICK IT) was for chess. It’s using a lot of brute force and some reasoning but it’s still not solving the same problem humans are and the domain is somewhat restricted.

Now the interesting difference is that whereas chess is a deterministic game where you can search for an optimal strategy, jeopardy has layers of uncertainty hidden behind human language and the behaviour of other players. So while it’s not a very realistic setting for general AI, and doesn’t claim to be, it has stepped over an important threshold from deterministic, logic based problems to ones that require reasoning under uncertainty and statistics.

This is very fitting as the field of AI research itself has gone through the same change in focus in the past 20 years as outlined very well by Peter Norvig recently. When I took my undergraduate AI classes in the 90s I fell in love with prolog and logical planning. That’s why I went into AI research later.

When I got to grad school I found out that during my undergrad AI courses I had been missing a renaissance that had been occurring which led to modern machine learning and probabilistic AI. Watson’s achievement is only possible with these new methods and the raw computing power increases we have had over the same period.

But  apparently it did also have one other advantage. As many people have speculated, the machine did seem to have a buzzer advantage. According to op-ed by Ken Jennings himself, Watson’s speed with the buzzer was decisive in making up for questions it got wrong.  Is this just sour grapes? Maybe just a little, you need some ego to be an intense competitor like Jennings, but I think he has a point. As I pointed out yesterday the quick reaction time between making the decision to buzz and registering a button press is something a machine can clearly be faster at. Is this what winning at Jeopardy means?

It shouldn’t be.

Winning should mean the ability to answer complex questions, with ambiguous meanings, under time pressure while making the best strategic betting choices.  That is the task Watson performed admirably at. It could have had a buzzer delay and read the screens with computer vision rather than receiving a text file to parse and perhaps it still would have won.

But we’ll never know now.

So you won this round Watson. And you’re impressive (well, the engineering team that built ‘you’ is impressive actually). Hopefully everyone has learned a bit about AI and hopefully some young girls or boys will be inspired to consider computer science or engineering that otherwise wouldn’t have.

But next year…next year you should come back and put it all the table.  Play it our way, the human way, you have the capability to at least try. And may the best machine, be they biological or electronic, win.

I’ll Take Spectacle for $1000 Alex.

News now that the next big public spectacle in the battle between Man vs Machine will be….Jeaopardy?

Update: more detail here

You may remember that computer’s have now defeated the greatest human players of chess inspiring endless punditry and loose discussion about ‘thinking’ machines as well as inspiring awesome Arcade Fire songs. Computers are also now quite good at playing poker, have solved Checkers completely (no point playing that anymore…), provide us with frustrating ‘automated phone help’ bots and regularly vacuum the floors of geeks fairly adequately.

Sigh. Perhaps this is why the New York Times article, which is otherwise pretty clear and non-hyperbolic about the next spectacle, felt the need to throw this in:

Despite more than four decades of experimentation in artificial intelligence, scientists have made only modest progress until now toward building machines that can understand language and interact with humans.

Now, I’m an Artificial Intelligence researcher, so I’ll try to be rational about this sentence.

The first half of the sentence refers to the common observation that four decades of research into AI has not produced walking,m talking androids trying to take over the world and consume us for power.  Instead it had provided tremendous research gains and advances in technology that underly  many aspects of our modern world from google to space probes, from self-driving cars to face detecting auto-focus cameras, from management of complex energy systems to medical diagnostic tools.  The second half the sentence points out that on the problem everyone on the street really cares about, walking talking androids that can ‘think’ like us and understand what we’re saying…that progress has been below society’s ridiculously high expectations.

Granted. Voice recognition has got a lot better over the years but not up to say, the 4 year old child level. But you know, we don’t even really understand how our own brains work, that makes simulating one in a less complex computing machine that the one between our ears, you know, tricky.  (A separate approach that may outflank current AI might in fact be just building an equally complex simulation of a brain and letting it go, but that’s another post.)

But I love these public spectacles, they provide a chance to explain the current level of AI and open up some of the ideas of computation in the problem that are used in more relevant applications all around us.  Having a computer up on t.v. with Alex Trebek and other contestants will be fun and we probably won’t even have the embarrassing situation Mr. Kasparov was in of the computer beating the human, not yet anyways.

It will be entertaining, some of it will be funny and hopefully some of it will be informative to viewers who live in an increasingly computational world.  Playing jeopardy well is a much harder problem than playing chess well.  The challenges it requires in terms of understanding language, meaning, searching databases, forming sentences and making strategic decisions about bids and questions are all very rich domains that have more real world application than the way chess playing programs work which is generally some kind of brute force search.

I just hope when the computer loses, the show is over and they ship the computer back to IBM labs we don’t hear another round of  “why such modest progress”?  This ain’t rocket science people, its a lot harder than that.

%d bloggers like this: