Don’t ask Nico Schlaefer what happens when “Jeopardy!” champions Brad Rutter and Ken Jennings take on Watson, IBM’s talking computer.
The Carnegie Mellon University computer science doctoral student who spent three summers working on Watson has been sworn to secrecy on the outcome of the taped shows airing next week.
“It is competitive. It could go either way,” said Schlaefer, a member of the Watson team that included fellow CMU researcher Hideki Shima and professor Eric Nyberg of the Oakland school’s Language Technology Institute.
Watson — named for IBM founder Thomas J. Watson — will compete for $1.5 million in prize money. Don’t be surprised if the game-show set looks a little unsettling when Rutter and Jennings take on the machine. Watson’s human competitors stand at podiums on either side of one bearing Watson’s name.
The enormous machine is not onstage. But every time Watson’s computer-generated voice buzzes in with a question to match one of the “Jeopardy!” board’s answers, a flat screen behind the empty podium lights up.
IBM team leader Donald Ferucci chose “Jeopardy!” to challenge Watson because the show features questions rife with puns, riddles, irony, metaphor and other language subtleties that often stump humans.
The software system provides answers to questions posed in language as it would be spoken. Although Watson is given questions in text format and provides computer-generated spoken answers, researchers say question-answering systems could be developed to address other formats such as speech or music.
While IBM relished the opportunity to show off its science to a broad audience, researchers walked a fine line.
“The fundamental challenge for IBM was to do something not simply viewed as media hype, but to open advancement of the question-answering initiative,” Nyberg said. He collaborated with IBM researchers and a team of experts from top research universities across the nation.
Nyberg will be available for online discussions at 10 tonight when PBS’ science show “NOVA” airs “The Smartest Machine on Earth,” an episode documenting the development of Watson. Schlaefer and Shima, who worked on the project during the summer of 2009, will be available for online chats sponsored by “NOVA” when Watson debuts on the game show Monday.
Nyberg was collaborating on the Watson project when Schlaefer arrived at CMU 312 years ago. The young doctoral student, who had developed his own question-answering system and put it out on the Internet for users to improve upon, was a natural for the project.
“The image people have of computer science is someone sitting quietly, alone in a lab, but that’s not the case today,” Schlaefer said. “A lot of people worked together.”
Schlaefer said the team faced the challenges of getting Watson to respond quickly and accurately. Like its human competitors, the machine would have to strategize and decide when its confidence level was such that not offering an answer would be preferable to risking an incorrect answer.
Shima said getting computers to evaluate risk has always been a problem but one on which scientists have made steady progress.
Schlaefer can remember when it took a prototype 30 minutes to answer a question.
“That would have made boring ‘Jeopardy!,'” he said.
Game show producers said Watson’s scores on the qualification test, as well as its performance in 50 test matches against former champions last fall, suggested the matches will be anything but boring.
“We’ve gone from impressed to blown away,” said executive producer Harry Friedman.
Although the computer can quickly sift through the vast stores of information programmers supplied it, Nyberg said Watson still lacks something its human competitors possess: a lifetime of collecting context while growing up in the real world.
“It’s going to make some mistakes that will be real howlers,” he predicted.
But if past is prologue, oddsmakers might want to take note of a prior man-vs.-machine match. When chess champion Garry Kasparov took on IBM’s Deep Blue computer in 1997 — another computer that boasted a number of CMU alums on the development team — Kasparov lost.
Click here to see Watson spar on YouTube.