Live, from IBM’s Thomas J. Watson Research Center: This is Jeopardy!
Today, IBM rolled out its Jeopardy-playing computer, a whiz machine named Watson that was four years in the works. In today’s demonstration match for the media Watson played against Brad Rutter and Ken Jennings, the two great (human) Jeopardy champions who will provide opposition for Watson in a two-day exhibition match. That man versus machine faceoff will air in February, and carries a prize of a million dollars. Bad news, humans: In today’s exhibition of about 15 questions, Watson tallied $4,400, compared to $3,400 for Jennings and $1,200 Rutter.
On stage, Watson was represented by a screen displaying its avatar (pictured) behind a podium, where a human player’s torso would be. Its avatar is a mobile graphic of the Earth with aurora-like lines swirling around it. When Watson was confident in its answer those swirls were shaded green; when it wasn’t they turned an orange hue. The questions were fed in plain text to Watson, but it had to wait the same amount of time to ring in as the human players did. To make the game fair, it also had to trigger a mechanical signaling button. Watson spoke in a stilted computerized voice–and was almost never wrong.
The machine started off on a roll in the category “Chicks Dig Me,” about women and archaeology. Jehrico. Agatha Christie. Mary Leakey. Crete. Watson fired off the answers so quickly it looked like it might blow its puny human competition off the stage. Fortunately for our species pride, Ken and Brad recovered with some right answers of their own.
After the demo, IBM trotted out Alex Trebek–to the crowd’s delight–allowing him to answer questions about the project alongside Jeopardy producer Harry Friedman and players Rutter and Jennings. Several IBM masterminds also weighed in, including David Ferrucci, who has led IBM’s attempts to make a machine that can play Jeopardy.
When one thinks of IBM and games, one thinks of Deep Blue and its legendary chess matches against grand master Gary Kasparov in the 1990s. But Ferrucci noted that “Jeopardy is a very different challenge from chess.” Chess is all cold logic, math, and strategy. Perfect for a computer. Jeopardy requires not only deep knowledge across all fields of human endeavor, but also an ability to parse and understand a question… and sort though the sparkling wordplay of Jeopardy clues.
The first part is easy for a computer. Ferrucci said his team fed dictionaries, thesauruses, plays scripts, and pretty much anything they could get their hands on into Watson’s databases (though it isn’t hooked up to the Internet, and it doesn’t understand foreign languages yet). The second part–getting a machine to understand conversational speech–is one of the grand challenges of artificial intelligence. While computer scientists spend their careers trying to push this field an inch forward, Jennings notes that even his 3-year-old can parse an English sentence.
Watson has already come a long way. Its setup is called “Massively Parallel Probabilistic Evidence-Based Architecture,” and it runs on 2,800 Power7 processing cores. If all that computer and interpreting power presents an answer that satisfies Watson’s confidence interval, it rings in. When IBM began the project in 2007, it tended to spit out nonsensical or embarrassing answers–when asked Ricky Ricardo’s signature song, for example, it would answer “song.” But over the last four years, as Watson’s algorithms have gotten better and better at blazing through its vast collections of data and figuring out how places, people, and other information is related in a written text (and what that means), its success rate has crept up toward that of the grand champions of Jeopardy: It buzzes in about half the time, and answers 85 to 95 percent of those questions correctly.
Tomorrow Jeopardy will tape the real encounter between Jennings, Rutter, and Watson. Are the human competitors nervous, especially since they haven’t actually played the game in five years? “Not now,” Rutter says, “but when Watson’s progeny come back to kill me from the future.” Jennings insists that he’s feeling cool. And setting aside his own pride for a moment, Jennings says it’s worth noting that humans built the thing. Whoever wins, we win.
Ferrucci is keeping his cards close to the vest before the big match, refusing to reveal too much about his prize quiz-fighter, like which categories might be his weakest, or his strategy in computing how much to wager on Daily Doubles or Final Jeopardy. As for Watson’s state of mind, Ferrucci puts it this way: “Watson does not have emotion, but Waston knows that humans have emotion.”
And Alex Trebek is just having a good time. When asked about his mid-game interview with Watson, the host noted that Watson can’t see or hear, so he will not be asking it any questions out loud. “I will probably make fun at his expense,” he says.
Related Content:
80beats: Your Clue Is: “This Robot Will Attempt to Crush Humans in ‘Jeopardy!’”
80beats: Watson, an IBM Supercomputer, Could Be the Next “Jeopardy!” Champion
Discoblog: First Chess, Now Poker? Computer Programmers Try To Crush Human Competitors
DISCOVER: Deeper Blue?
January 13th, 2011 at 2:44 pm
I’m surprised they don’t require the computer to take voice input from the host reading the question, one of the advantages a human may have would be to answer questions before they’ve been fully spoken..
January 13th, 2011 at 3:13 pm
Unless they changed the Jeopardy rule, the players are not allowed to buzz until the question was completely read by the moderator. I remember reading (when Ken Jennings was ruling the stage) that one advantage Jennings had was experience (from having played so many days in a row) with the timing system for the buzzer. On most questions, contestants who figure out the answer will have done so before the question was completely read. So having a feel for when the buzzing window opens can give the player an edge.
January 13th, 2011 at 3:13 pm
@ Dr. Kenneth
One of the rules of Jeopardy is that contestants cannot buzz in until the host has finished reading the entire question. I do think it’s an unfair contest in that the computer is getting the clues in text only. The computer should be required to act on the spoken question.
January 13th, 2011 at 3:15 pm
@Bill Excellent point; they say people watching at home often drastically underestimate how important buzzer timing is to winning the game.
January 13th, 2011 at 3:18 pm
So, if Watson had to do spoken text only, would you blindfold Ken and Brad so they couldn’t read either?
Conversely (contra-positively?), are deaf people not allowed to be on Jeopardy!? Because they can’t hear the spoken question. I know there was a blind champion a while back…
January 13th, 2011 at 3:26 pm
Eddie! I’m always surprised he’s not mentioned in the same breath as Ken Jennings. It seemed that Eddie could have been a champ for a long, long time, but that was before the rule change.
I’m excited to see this episode- best believe I’ll be tuned in in February!
January 13th, 2011 at 3:33 pm
‘As for Watson’s state of mind, Ferrucci puts it this way: “Watson does not have emotion, but Waston knows that humans have emotion.”’
This is a genuinely creepy thing to hear.
January 13th, 2011 at 3:57 pm
In college I went to a talk by a 5-time Champion (the limit at the time), and he specifically mentioned the buzzer timing. There is a guy sitting off stage that enables the buzzers once Alex finishes reading the question, the contestants can literally see him flip the switch. He said he always used the first few low-dollar questions to practice his timing while watching the guy enable the buzzers.
January 13th, 2011 at 5:46 pm
I for one welcome our new elementary overlord
January 13th, 2011 at 8:17 pm
So, does Watson put the responses in the form of questions? How does it handle who is/what is choice?
January 13th, 2011 at 10:50 pm
Watson’s real advantage is that it actually knows the clue before the human competitors do. Although Watson might only receive any actual clue at the same time as the other competitors, the human beings have to take a moment to actually read it — Watson can spend this time cognizing what the question means and cross referencing its database for possible answers before the human players even finish knowing what the question actually was, which can give it a significant advantage. My guess is that Watson performs much better against humans when the clues are more verbose and thus take more time for the human players to finish reading.
January 13th, 2011 at 11:29 pm
@TerryCloth: Yes, in the form of questions. And for your second question, there is probably 3 years worth of artificial intelligence research required to explain it. :D
I’m disappointed you didn’t have a picture of the whole Watson monolith – except for the display it’s a dead ringer for the monolith from 2001….
January 13th, 2011 at 11:38 pm
The computer is basically just doing a fast lookup from the 10 million books and such entered into its memory. Also, Watson utilizes 2,880 “brains” or processors, while each human player uses 1. Not to mention each human player is mobile and doesn’t take up the space of 12 refrigerators.
January 14th, 2011 at 2:19 am
I don’t think the computer just does searching and answering things.Each answer from the computer depends on context analysis, so it makes a great step for artificial intelligence
January 14th, 2011 at 2:48 am
@John Each of those 2,880 processors is not the equivalent of a human brain. Indeed, the human brain is not ‘single’ in the same sense each of those processors is. There is parallel processing that occurs in the neural system.
January 14th, 2011 at 3:33 am
I wonder what Watson will formulate as a question to the answer ’42′.
January 14th, 2011 at 7:50 am
@Johan: What is the answer to Life, the Universe and Everything from the book The Hitchhiker’s Guide to the Galaxy?
(It’s a very easy question to answer outside the universe of those books.)
January 14th, 2011 at 10:29 am
“42″ could also be “What uniform number belonged to Jackie Robinson?” – Again, context would be key. Then again, there are many questions for which the answer would be “42.” Though, for any true, ultimate answer, the one supplied (Life, the Universe…) would have to be accepted regardless of the category given…
January 14th, 2011 at 11:41 am
” ….almost never wrong. ” What a terrible statement. Why not say it got one or two wrong. Almost never, oxy-moron.
January 14th, 2011 at 3:01 pm
@Nibra — “always never” would be an oxymoron (no hyphen). “Almost never” is perfectly expressive and not in any way a terrible statement.
January 14th, 2011 at 7:00 pm
This is really quite impressive. Parsing normal human speech and phrasing has been very difficult for computers. They struggle with ambiguity, missing information, probabilistic assignments to meaning/intent, and so forth. Even with textual input that only helps with the actual speech to text component.
Also, coordinating the knowledge contained in Watson’s databases must be a challenge. Merely having the data does not mean that it is easily accessible or usable.
January 16th, 2011 at 1:29 am
Correct me if I’m wrong, but won’t the humans be fighting to split up the points that can be accumulated against the machine? If Blue Boy get 35% of the points available….he wins if one of the humans can not defeat the other. Bit of a skewed contest….how about trying it one on one?
February 18th, 2011 at 11:04 am
Did Watson have a switch that he had ot flip before he answered?