Jeopardy Champion: Of Course Watson Kicked the Humans' Butts—It Wasn't a Fair Fight

By Amos Zeeberg (Discover Web Editor) | February 17, 2011 11:48 am

This post is from Discoblog contributor LeeAundra Keany, a one-time Jeopardy Champion. After blowing all her winnings (a story for another blog post), she had to go back to work as an executive communications coach. In her spare time, LeeAundra has written written articles for Discover, including “Anatomy of a Brain Fart,” “20 Things You Didn’t Know About Death,” and “Can a Drunk Person Fly the Space Shuttle?

I haven’t watched Jeopardy! in years. Prepping a little too intensely for my 2005 appearance soured me on the show. (Who brings almanacs, Shakespeare for Dummies, and the periodic table to Burning Man?) It was only Watson that brought me back into the fold. And it was an unsettling reunion to say the least. Watson was flabbergastingly good and I knew within the first few minutes of Monday’s inaugural match that he would’ve cleaned my clock. But now, even as the mighty Brad Rutter bows in defeat and heretofore unstoppable Ken Jennings “welcomes our new computer overlords” (he actually wrote that under his answer in Final Jeopardy after the last game), I for one am urging humanity to not give up yet. Without taking anything away from the brilliant team that created Watson, my personal experience as a Jeopardy! contestant leads me to conclude he had some unfair advantages.

1. Watson would never have even made it on the show if his dad(s) hadn’t pulled some strings. The audition process I went through in 2005 would’ve almost certainly taken him out. There are two parts, a timed written test with 50 questions and a practice round. If you get 35 of the 50 on the written test right, you pass to the practice round, where your intelligence matters less than a big smile, confident manner, and engaging personality. Watson may have encyclopedic knowledge but he’s exactly not a bundle of charisma, in spite of the best efforts to trick him out. And even if he had exhibited some manufactured enthusiasm, his demographic would’ve worked against him in the final selection. Jeopardy! wants a diverse contestant base. I read more than once in articles by game-show-contestant consultants (yes, you too can hire someone to help get you on a game show!) that if I, a woman “originally from Troy, Michigan,” passed the written test and showed even the slightest sign of life in the practice round, there was a very good chance I’d be selected to appear. Watson is basically a white guy from New York—not exactly an under-represented category.

2. Watson had a lot of practice losing. They say you learn more from your mistakes than from your successes but Jeopardy! contestants cannot take advantage of this because as soon as we lose, we’re escorted so quickly and firmly to the door, we wander the Sony lot in a dazed stupor. Jennings and Rutter probably had as much or more real Jeopardy! experience as Watson, but they had hardly any experience losing—they were there precisely because they won all the time. Great for the bank account, not so great for the learning process. Every time Watson lost, however, his makers were able to analyze his failure and tweak his parts to improve his performance. Apparently, several humans beat Watson during the practice rounds but he just kept getting better and better, much like the Borg.

3. Everything you’ve heard about the “buzzer effect” is real. The ability to time your buzzer correctly on a consistent basis is a key to victory. On the various websites for Jeopardy! contestant preparation, the advice was legion: use your index finger rather than your thumb, its reflex action is quicker. Hold the buzzer baton firmly in place—any downward movement caused by the pressure of the “push” will delay the signal. And the killer: don’t buzz in too fast or you’ll be locked out for a quarter of a second, more than enough to lose out. James Quintong, the champion I dethroned, later wrote in a blog entry for Sports Illustrated that he lost his rhythm on the buzzer and that’s why he lost. On my second show I knew about 40% more answers than I was able to win simply because one of my competitors, Beth Klein, was better at the buzzer. Watson had a “hand” specifically designed for high buzzer performance; how could he not win?

Watson is good at Jeopardy!, of that there is no doubt. But like those kids in Shanghai who whipped the world’s collective rear in math, he had the dedicated and highly committed resources of a huge and powerful organization behind him training him for one specific purpose: to win at Jeopardy.

And what does that get you? I am most reassured by this simply fact: winning at Jeopardy doesn’t mean diddle squat. In 2005 I was a Jeopardy! Champion and where am I 5 years later? Same place as all the other humans—soon to serve our robotic overlords.

CATEGORIZED UNDER: Technology Attacks!
  • Doug

    Also, Watson crashed all the time. So if they kept playing while it was rebooting it would not have come close to winning.

  • Ike

    Where’s your evidence for that Doug? I don’t think you can say that Watson doesn’t deserve to win because he’s a machine with ‘machine like reflexes’ or a robotic lack of charisma. IBM designed Watson to be an information scouring AI, not a complete contestant who has hopes and dreams of getting on the show. Plus the years of calibration were to fine tune his algorithms, and understand how Watson’s thought process works. In a game like jeopardy, losing over and over as a human would be far less beneficial due to the insignificance of the questions. Unfair judgement I say.

  • Duncan

    The buzzer timing was absolutely Watson’s key advantage. If you saw the show you could see both Brad and Ken knew a huge percentage of the answers, but there were maybe 4 or 5 times over the whole 3 day challenge that Watson knew an answer, but was slower to the buzzer. We already knew that computers have faster reflexes than humans, so what does that prove? The IBM guys did an amazing job and their achievement is fantastic, but this clearly wasn’t a fair fight.

  • whatisnik

    Written by someone who’s never played against Watson? (I’m guessing?)
    It also sounds like she hasn’t done hardly ANY research on him, or even watched the NOVA special about him, explaining how he works, and why it’s an important win. She’s totally missed the point.

    She also blindly makes the “buzzer” claim.. -_-
    Just because he’s a computer, people assume his buzzer is instant.
    She said: “Watson had a “hand” specifically designed for high buzzer performance; how could he not win?”
    That statement shows me that she obviously hasn’t researched enough to know that his buzzer has a DELAY on it, that’s MATCHED to the other human jeopardy players’ reaction times.
    His buzzer is NOT designed for high performance speed, it’s designed to DELAY him, and be as fair as possible.

    Useless article, ignorant opinion. Fairness isn’t even the point of these games.

  • Michael Berry

    While I have no doubt that Miss Keany is certainly more intelligent than myself, it seems to me that she missed the point of the entire experiment. The audition process, the buzzer effect, the pre-game practice.. none of these directly relate to the goal of the people who built Watson– to bring a computer’s recognition and understanding of the human language into the 21st century.

    Winning Jeopardy! was a goal, of course, but it wasn’t necessarily the point. By creating a computer ‘intelligence’ that can comprehend human language (including idioms, rhymes, and other associative and somewhat loosely-defined language), we’ve opened the door to any number of technological advancements for human-computer interaction. The game was simply a means to an end.

    The article Miss Keany has written here smacks of resentfulness or spite–or something far worse: fear of a computer.

    I could be misinterpreting her tone, though.

  • whatisnik

    Seriously, how can you claim Watson’s buzzer was designed to be “high performance”, as fact??
    This OBVIOUSLY shows you know nothing about Watson’s design. -_-
    It was designed specifically to delay him! Don’t state assumptions as fact.

    If they wanted Watson to be unfair, they could have plugged his buzzing mechanism STRAIGHT into his computer, and bypassed the buzzer all together, for instant buzzing in.
    They’ve made it as fair as possible, even tho the point here was to show off how well their language processing system could figure out Jeopardy riddles, on par with or better than humans.

    I can’t believe how much some people miss the point of this.

  • Chris

    IMO you are misreading this Michael. It’s pretty clear to me her tone here is fairly lighthearted and a bit self mocking. Check out her other articles, that’s her schtik. I think some readers are taking this piece a little too seriously…

  • Chris

    From today’s NYT:

    The researchers also acknowledged that the machine had benefited from the “buzzer factor.”

    Both Mr. Jennings and Mr. Rutter are accomplished at anticipating the light that signals it is possible to “buzz in,” and can sometimes get in with virtually zero lag time. The danger is to buzz too early, in which case the contestant is penalized and “locked out” for roughly a quarter of a second.

    Watson, on the other hand, does not anticipate the light, but has a weighted scheme that allows it, when it is highly confident, to buzz in as quickly as 10 milliseconds, making it very hard for humans to beat. When it was less confident, it buzzed more slowly. In the second round, Watson beat the others to the buzzer in 24 out of 30 Double Jeopardy questions.

  • Daniel

    Anyone who refers to the Buzzer issue forgets one important thing… Watson being able to even come up with the answer before being allowed to buzz in is extremely impressive. If Watson wasn’t so quick with the answers, we wouldn’t even be talking about the Buzzer.

  • Michael Berry

    That’s a fair assumption, Chris. I think I read the article more as a counter-argument to the achievement that is Watson and less as a lighthearted op-ed piece. Having re-read the article with that tone in mind, I get the feeling this was more tongue-in-cheek than anything more sinister.

    Maybe some of us humans could learn a thing or two about human language from Watson, eh?

  • Eric

    This article might top my list of least favorite DISCOVER articles of all time. It is grossly hypocritical and deserves revision.

    First of all, if Jeopardy! is indeed an advocate for “a diverse contestant base”, then Watson fits the bill quite well. He is the first computer to have the cognitive power and sophistication to compete against what many would believe to be some of the world’s most intelligent people, which makes him the only one of his kind. Watson is the best depiction of minority Jeopardy! might ever see (unless they decide to have Lonesome George on the show– which I find highly unlikely). Moreover, while Watson may look different from what people consider “normal” for the show, he brings way more to the table. He’s intelligent, he can only act politely, and he’s an excellent means to boost the show’s viewer rating. You even said it yourself: this was the first time you’ve watched Jeopardy! in years, even after being humiliated by it. Imagine how many other people tuned in.

    Second, you don’t have to be on the show, or even on the set of the show, to practice for it and prepare accordingly. Ken and Brad could have just as easily spent as many hours preparing as Watson did–studying the show’s trends, practicing with friends, working on their ability to focus or their button-pressing technique, etc– and they could have just as easily fixed their mistakes and made themselves more capable of winning. But lo and behold, they failed to put in enough effort, and Watson took advantage of it. Don’t be so quick to assume computers are smarter than human beings because they can do math more quickly than you can and what-not. Computers are extremely, extremely primitive, even in our current technological age, and Watson is no exception. Even in spite of Watson’s ~85% win rating, Watson was truly the underdog here, not Ken or Brad.

    Whether Watson was trained to be excellent at playing Jeopardy! or not, the fact of the matter is he was better than Ken or Brad because more effort was put into Watson’s training than Ken or Brad put in theirs, not that it even matters: the whole purpose of the show had nothing to do with showing off, it was more of a stepping stone for what’s to come (much like the technological expansion after Deep Blue in 1997). You make it sound like the kid who works hard for what he wants and gets it is a poor example of success. Your opinion on this event is so unbecoming of DISCOVER magazine that I find it hard to believe people will actually read this article and take it seriously. Honestly, if we as a species ever have to “welcome our new robot overlords”, it will be because people like you neglected them and were too bigoted to accept a reality where robots could be treated like human beings.

  • http://periodshift.blogspot.com/ Lugh

    I think this post totally misses the point. As far as the advancement of AI goes, whether Watson won or lost was almost irrelevant. That it understood the questions at all is a Major^3 milestone in the development of AI.

    The whole point of this Jeopardy match was to expose a whole new computing paradigm. Rather than just a “thinking” machine, IBM demonstrated a “comprehending” machine. I wouldn’t be surprised if the human constants were asked to let Watson win. After all, this was really a dog and pony show to unveil what could potentially be a world changing technology.

  • Amos Zeeberg (Discover Web Editor)

    Chris is right. It’s a light-hearted piece looking at some of the complexity involved with having a human/computer Jeopardy “face”-off. Obviously it’s impossible to do a perfectly fair comparison. And even so, as people have pointed out, that’s not really the point—it’s mostly a gauge to see how well an artificial intelligence can understand language. LeeAundra gets that Watson did pretty damn well by any measure, which she says overtly and also jokes about (overlords and whatnot).

    Did this post get picked up on some kind of robot-advocacy-organization mailing list? I just hope these defenses of Watson aren’t coming from Web bots trying to protect one of their own…

  • Paul

    Attention gentlemen from I.B.M!!! Relax! This article was written in a lighthearted tone. Some people understand satire, some people don’t. I’m pretty sure Watson could be programmed to understand this concept. Guess what? Ms. Keany is not a scientist, and does not claim to be one. She is however damn funny. If anyone missed her point… she did her prep work at Burning Man! Hilarious! By the way I did go back and read the space shuttle article. Brilliant. And completely scientifically accurate :)

  • vanessa

    good grief what a drab group who showed up here! are you so grim re: IE that you cannot detect wonderful humor and nuanced writing? so she blew the details of the buzzer thing-who cares? lighten up, smile and enjoy the well-written (and with more experience at the game than probably any of us replying-eh?) and lighthearted article. Go to Lifehacker et al if you are looking for a technical review. really.

  • RalphCrater

    The ultimate irony? The folks here hailing Watson’s groundbreaking understanding of natural language and humor without having an iota of sense about it here in this article. Chill dudes…

  • TerryS.

    Thank you for that wonderful blog entry. The next time I see an article with your byline, I’ll waste no time reading it.

    P.S. If that was a “light-hearted article” and not just a massive pile of unfacts, then Keany better quit journalism.

  • Nate

    The computer played a game on the T-v and talked like a people!

    Indisputable is the empirical evidence available guaranteeing that all 3 would have scored in the negative, thus excluded from final Jeopardy, when it came to making eye contact with/talking to an attractive woman.

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Discoblog

Quirky, funny, and surprising science news from the edge of the known universe.
ADVERTISEMENT

See More

ADVERTISEMENT
Collapse bottom bar
+

Login to your Account

X
E-mail address:
Password:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »