But Google’s co-founders, Larry Page and Sergey Brin, wrote something even harsher in The Anatomy of a Large-Scale Hypertextual Web Search Engine (1998):
How intelligent are the Computers of 2011
True, the computer system Watson, created by IBM beat all human competition in the US reality quiz show Jeopardy! But does it make Watson intelligent – is it an important step towards Singularity? Is it a significant step in the field of AI? Or, is the win meaningless and does not imply anything significant? What is the future of this technology? How far are we from understanding how the brain works?
If you are not familiar with Jeopardy! Then the best place to look is, where else, Wikipedia page. Let me just mention here that the open source community is quiet excited about this win as the software component of Watson relies heavily on open source software – on Apache, Linux, Hadoop…
However, if you want to know how Watson works this is probably the best post – Could Google Play Jeopardy Like IBM’s Watson? – written by Danny Sullivan. Also, here are some thoughts on what Watson can be used for in the future – Envisioning IBM’s Watson computer as a Smartphone app.
Is the core technology inside Watson significantly ahead of the others?
How well does it compare with Google? What about Microsoft? What about the semantic question answer engine – Wolfram? Here is an interesting post by Stephen Wolfram – Jeopardy, IBM, and Wolfram|Alpha.
To quote from the post:
… just feeding Jeopardy clues into it, and seeing what documents get matched. Well, just for fun, we tried that. We sampled randomly from the 200,000 or so Jeopardy clues that have been aired. Then we took each clue and fed it as input (without quotes) to a search engine. Then we looked at the search engine result page, and (a) saw how frequently the correct Jeopardy answer appeared somewhere in the titles or text snippets on the page, and (b) saw how frequently it appeared in the top document returned by the search engine…”
There is no way to tell if Watson is a giant leap forward or it is so tailored for Jeopardy! that it is of not much direct use elsewhere. The opinion in the blogosphere is divided. Personally, I find it difficult to believe that even if Google does not have the technology today, it will not take them more than 6-9 months to match or exceed it.
Why has Google not attempted something like this?
Here are the possibilities:
- The technology is indeed very advanced and Google cannot easily replicate it.
- Wait for few months and Google will have something similar in place.
- Google does not perceive any business value for doing this – after all it is already close to 70% accurate in retrieving the correct page.
- The extra computation needed to arrive at the answer will be too much and the speed of response will suffer.
- Giving an answer instead of the links will adversely affect the advertisement revenue.
- The technology inside Watson is too Jeopardy! specific and cannot be easily generalized.
Take your pick!
Is IBM only interested in selling hardware?
This TED talk gives you some insight into Watson. The participants are Steven Baker (author of Final Jeopardy!), Kerrie Holley (IBM Fellow looking for Watson’s next job), Dr. Herbert Chase (Columbia University Professor of Clinical Medicine) and Dr. David Ferrucci (IBM Watson Principal Investigator).
Dr. David Ferrucci talked sense. According to him about 20-25 people worked for 4 years to bring Watson to what it is today. The answer I really like was his reply to the question “Does Watson think?” His response was “Do submarines swim?”
However, I cannot say the same for what Kerrie Holley said. He was asked what “Watson can be used for?” His response left me speechless! How can somebody who is in charge of looking for Watson’s next job make such naïve statements? How can he be so misinformed about what Watson is? Read it for yourself – I have transcribed his reply here:
Business Analytics is a huge area. Improving decision making … making fact based decision … evidence based decision. This is something that C-suite executives are going to love. If you think what PCs have done to enable and make them more effective … think about what internet has done. I think this represents the next stage of evolution because suddenly they have the C-suite assistant and they can begin to ask questions like if I do this if I do that what impact it is going to have on my profit, what impact will it have on my revenue? If my company is under siege financially and I have to make some cuts in my marketing budget, do I cut what I spend on TV, do I cut the print. Suddenly I can ask these questions … not only do I get an answer and probability but also get the reason why the answer of why those answers are of 80% probability or 70%. So that is an example of C-suite assistant … this is going to be a powerful tool … based on analytics … based on all the technologies you see in Watson. The brilliance of human is the ability to ask questions … that is what the machine does not do right now … so it is a great complement.
If you look at city planners who are planning traffic who are dealing with congestion … dealing with weather … dealing with accidents … this could provide another tool to planners to make better decision dynamic real-time decisions of routing traffic. Look at it as a navigator on steroid … whether it is a smart phone … whether it is a navigator on a car … we also have this capability as well where we can ask questions about we need to get here in 5 minutes … there is an accident in a road right now … what do you recommend? I think that is a powerful capability as well. Also from a travel and entertainment standpoint our ability … may be I am stuck in Alaska and I need to get into London … whatever the situation is … now use this technology to get me out of the bind … but may be to recreationally to help me plan the most optimum trip based on my preferences … based on what is happening in the world.
Give me a break! Is Watson an analytics engine? Is it a GIS software?
IBM says the technology is available starting at a million dollar which includes hardware and software. Details are not yet available on what you will actually get for this money – let us wait and see. I am also curious to know if the push for healthcare industry is because of the federal spending.
Also will there be a new dot on Gartner Hype Cycle for Emerging Technologies next year?
Feb. 18, 2011, 1:05 p.m. EST
Watson is just a super search engine
Commentary: Pure algorithmic search is the holy grail
Stories You Might Like
You might like:
- The clouds begin to part
- David Weidner’s Writing on the Wall: Sorry…
- PVH’s quarterly results in after-hours focus
- Gold futures give up early gains
By John C. Dvorak
BERKELEY, Calif. (MarketWatch) — If we learned one thing by watching the Watson computer challenge real humans on the “Jeopardy!” game show, it’s that International Business Machines Corp. has developed a new kind of search engine.
That’s what this is all about, a search engine, probably called Watson that will obviously take straight English queries and deliver answers. This has been the promise of numerous search engines, beginning with Ask Jeeves. That idea is still being explored by Ask.com, owned by IAC/InterActive Corp. /quotes/zigman/111865/quotes/nls/iaci IACI +0.31%
This Watson device should go online immediately — assuming it actually works and is scalable.
Obama talks tech with Zuckerberg, Jobs
President Barack Obama dines with the technology elite, including Facebook’s Mark Zuckerberg, Oracle’s Larry Ellison, and Apple’s Steve Jobs in San Francisco. Video courtesy of Fox News.
All we know so far is that IBM /quotes/zigman/230066/quotes/nls/ibm IBM -0.06% intends to produce a medical-expert system using the technology.
During the heyday of the AltaVista search engine and the early days of Google Inc. /quotes/zigman/93888/quotes/nls/goog GOOG +0.30% , IBM had been playing around with search technology and had an internal “private” engine that was as good as anything commercially available. I followed its development closely, and somewhere along the line IBM dropped the idea.
But this indicated to me that the company had some interest. Surely the attention (and money) given to Google must have been noticed by now.
When I think of the possibilities, I wonder if the algorithms can be used for more than trivia but to determine the kind of websites that people are looking for. In other words, can these algorithms deliver the same kind of results as Google?
I don’t see why not.
Pure algorithmic search results without needing to cache the entire Internet every few minutes is a holy grail of these technologies. At some point, Google’s cache’s will be larger than planet Earth. If you want to stop global warming, stop Google from adding more server farms.
I cannot find anyone at IBM to comment on whether the company has any immediate plans in this area. Watson is a research project; IBM does a lot of research.
That said, we can assume some interesting patents will evolve from this device. We can further assume that at the next IBM shareholders’ meeting, any number of investors will ask executives why the company is not using Watson for a search engine to compete with Google and Bing.
The irony is that if Microsoft Corp. /quotes/zigman/20493/quotes/nls/msft MSFT +1.51% had kept better relations with IBM during the OS/2 vs. Windows era, Microsoft could end up with Watson and use it with Bing in some sort of joint venture.
Now that would give Google a real run for its money.
Right now, any number of companies would benefit from a deal with IBM for the Watson search engine. Ask.com looks the most obvious. A long shot but interesting possibility is WolframAlpha. Visit computational engine WolframAlpha.
In the meantime, IBM will continue to experiment with subtext versions of the engine, such as the medical-expert system.
But if the company can manage to promote a new search engine, it has the opportunity to make billions in additional revenue in the search market dominated by Google, with Microsoft’s Bing and Yahoo Inc. /quotes/zigman/59898/quotes/nls/yhoo YHOO +1.17% bringing up the rear.
Of course, for all we know the whole “Jeopardy!” challenge may have been staged in such a way to make Watson look better than it is. We just don’t know.
I got suspicious when Watson added five question marks to the wrong Final Jeopardy answer in the second show.
That made no sense to me. But we’ll see. I hope it does work as advertised.
Former tech editor of BusinessWeek
Posted: January 8, 2011 10:00 AM
share this story
Get Technology Alerts
Submit this story
While working on my book about IBM’s Jeopardy-playing computer, the most common question I encounter is this: Doesn’t Google already answer questions?
The short answer is no. Google depends on our brains in two ways: It gets us to think like a computer when formulating our query, picking three or four words that will make most sense to the machine. Then it directs us to the neighborhood of the answer we’re looking for, but leaves it to our infinitely more nuanced brains to find exactly what we’re looking for there.
Watson, which will face off against two Jeopardy legends, Ken Jennings and Brad Rutter, in February, has to handle all that work by itself. It must decipher complex English, hunt down possible answers, choose one, and decide if it has enough confidence to bet on it.
Here’s an example: “When 60 Minutes premiered, this man was U.S. president. That’s a tough one for a computer. It has to understand what “premiered” means and that it’s associated with a date. Then it has to figure out the date when an entity called 60 Minutes premiered, and then find out who was U.S. president at that time. In short, it requires a ton of contextual understanding — or a statistical simulation of it — and then two different hunts, one for the date, the second for the president.
Once Watson has a list of possible answers (or “responses,” as they call them on Jeopardy!), it has to figure out which one merits the most confidence, and if it’s sure enough of the answer to place a bet on it. All these takes place in about 3 seconds. (By the way, the answer is Lyndon Johnson.)
Watson has more than 100 algorithms leading it to solve these Jeopardy clues. Each one has its specialty. One of them helps it with specifically this kind of question. It’s called “nested decomposition,” and it involves breaking the clue into two different hunts. This may sound really obscure, and only marginally useful. But if you listen to people asking questions, a lot of them require this type of hunt. “What’s the best pizza joint near campus? Which southern state has a big steel industry? etc.
So you might think that Watson could become the next Google. One big problem. To solve that clue, Watson uses more than 2,000 processors and consumes loads of electricity. Google, in those same three seconds, responds to millions of search queries. Google uses perhaps one-billionth of Watson’s resources, or less, to handle each query. So the two approaches don’t compete. But in coming years, Google and the other search engines will start to answer questions, more like Watson. (They’re starting with simple ones. Nested decomposition is still a ways off.) And to get Watson jobs outside of show biz, IBM will have to figure out how to run such machines for a fraction of the cost.
Follow Stephen Baker on Twitter: www.twitter.com/stevebaker
Is IBM’s Watson the ‘Perfect Search Engine’ Google Has Been Talking About?
http://api.tweetmeme.com/button.js?url=http%3A//www.screenwerk.com/2011/02/18/is-ibms-watson-the-perfect-search-engine-google-has-been-describing/&&source=gsterling&style=normal&b=2&o=http%3A//search.yahoo.com/search%3Ftype%3D61107%26fr%3Dfreecause%26ei%3Dutf-8%26p%3Ddoes+google+want+IBM+watsonI’m probably the only person who didn’t watch the amazing defeat of the human Jeopardy champions by IBM’s Watson. The whole affair was a brilliantly “engineered” commercial for IBM. The company has already received contracts from Nuance and others after the win.
But after reading all the stories and watching all the video my belief is that Watson’s moment is a dramatic milestone that will help transform information retrieval and decision-making, which we now call “search.”
Danny Sullivan writes a spirited defense of Google (vs. Watson). However in my view what Watson represents — understanding and “right answers” — is a successor to Google and its approach, which is probabilistic.
Siri, which Apple acquired last year, was an effort to take users from query to “transaction,” to bypass links and get them to actionable information or task completion. In other words, Siri’s objective was to narrow choice rather than expand it. This is in fact the evolutionary, stated objective of search engines like Bing (and Google) today: “answers not links.”
Google’s Marissa Mayer has said many times that the “perfect search engine” is one that “understands” what we want:
It would be a machine that could answer that question, really. It would be one that could understand speech, questions, phrases, what entities you’re talking about, concepts. It would be able to search all of the world’s information, [find] different ideas and concepts, and bring them back to you in a presentation that was really informative and coherent.
Let me assert that the future of search is mobile; smartphones outsold PCs in Q4 of last year. Indeed, there will be more “connected devices” in use around the world than PCs very soon. Mobile devices will be used everywhere. However PCs are used in much more limited circumstances: at your desk, in a conference room, at the cafe and so on.
The generally smaller screens and “on the go” use cases surrounding mobile devices make more precise and targeted information more important than millions of links. Because of information overload and limited attention, we now only care about the top half of the first page of search, what is “above the fold” as it were.
People want three choices, not 10. When I ask for “the best sushi restaurant in New York” I want a very few of the best options not 390,000 links.
In the search result above Google does provide a narrowed list but there’s still lots of “waste” in these results. On a PC I can look at all the sites, assess their credibility and make my choice. On a mobile device I don’t want to spend the time sifting through links.
Today people are using conventional search algorithms, popularity lists and social techniques to try and filter the overload of information. In parallel they’re also using some of these methods to help me “discover” new places and new information. In general however social media tools, recommendations and reviews are all about helping make more informed choices more efficiently.
Since I’m not an engineer I can’t speak with any authority about the technology behind Watson and how it can or cannot be applied in other areas. However, its apparent capacity to “understand” and respond to complex queries with a single answer strikes me as a successor technology to what we have today.
Will Watson Win on Jeopardy!?
- Posted 01.20.11
A computer taking on the best human contestants on Jeopardy!?—the prospect is perhaps even more astonishing than when IBM’s Deep Blue computer challenged the world’s top chess player in 1997 (and won). NOVA recently asked three experts on artificial intelligence about Watson, its capabilities, and its implications beyond the game show. They include David Ferrucci, head of the IBM team that programmed Watson to play on Jeopardy!; Rodney Brooks, roboticist and recently retired head of the Computer Science and Artificial Intelligence Laboratory at MIT; and Luis von Ahn, a computer scientist at Carnegie Mellon and an inventor of CAPTCHA, the squiggly words and numbers used in computing to ensure a response is not coming from a computer. The following is an edited version of that conversation.
A mock contest between Watson and human contestants on Jeopardy! When the real contest occurs, how will the supercomputer fare? Enlarge Photo credit: Courtesy of IBM
WATSON AND JEOPARDY!
How is Watson so successful with understanding natural language?
David Ferrucci (Research Staff Member and leader of the Semantic Analysis and Integration Department at IBM’s Thomas J. Watson Research Center): I think what’s making Watson successful is its internal architecture. It’s looking at so many different algorithms—thousands of different algorithms—some of them focused on understanding the question, weighting the various terms, looking at the grammar, the syntax, finding the phrases, the keywords, the entities, the dates, the times, trying to understand what it is being asked. And this, in itself, is a big challenge, where we use a variety of different technologies. But ultimately, what’s exciting about it is how it looks at many, many different possibilities and assesses them and builds confidence in a final answer to decide whether or not it’s correct and whether or not it wants to risk buzzing in on Jeopardy!
Why did you guys choose Jeopardy!? What about this game show in particular makes it the ideal challenge for a computer like Watson?
Ferrucci: Well, Jeopardy!‘s a really fascinating game that challenges a computer to deal with language, to look at an unusually broad domain. You can’t anticipate ahead of time that there will be a question about a particular single topic, for example, or that it’s going to be phrased any one way. So you have that breadth. You also have to have precision. You have to have the right answer in the top spot. No points for second or third place or somewhere in the top 10 documents. You have to have that confidence. You have to know that you’ve got it right, otherwise you don’t want to risk buzzing in, and you have to do it really, really quickly. So all of these aspects of the game help us push that natural-language-understanding technology in a way that we’ve never really been able to do before.
What happens if Watson crashes during the taping of Jeopardy!? What will you guys do?
Ferrucci: Well, we cut tape—is that the expression they use?—and we see if we can bring it back up. But Watson is complicated. It’s essentially 10 refrigerators’ worth of hardware. There’s about a million lines of new code in there. So it’s a complex system. We’ve done a tremendous amount of testing, but anybody who designs software knows something could always go wrong. And we’re prepared to fix it and bring it back up rapidly. We’ll see.
So what implications does Watson have beyond competing on Jeopardy!?
Ferrucci: We looked at Jeopardy! as a challenge that drives the technology, not, obviously, as an end goal. And we kept a careful eye to focus on a reasonable capability, something that ultimately will give us the ability to look at huge volumes of text, do a better job at understanding them, and pull out the information that humans are looking for, the precise facts, the precise opinions, whatever it is that you’re asking about, to do a deeper understanding of what you want and being able to get that breadth and that precision over language to get you the right answers. And we’ve already seen that helping. We’ve already started looking at applying this technology to a number of different areas, including medicine and healthcare as well as text support, publishing, finance. We’re actually very excited about some of the preliminary results.
“Whether it wins or not, to me, is irrelevant. It’s being able to be in the game that is the really big advance.”
What about for AI and computer science? What is Watson’s promise for those?
Luis von Ahn (Professor of Computer Science at Carnegie Mellon University): I think there are a lot of implications, not just for AI and computer science but also for the world, for things like Google. Right now on Google, all you do is you type in some keywords, and it gives you links, but imagine if it could start answering your questions, as opposed to you having to go and find the answers. It’s starting to do that. For example, if you go to Google and ask it, “What time is it in Austria?” it will tell you, but those are very simple questions. The implications of this is that it can give answers to much more complicated questions. I mean, it’s probably not doing this yet, but you ask it things like “My head and my feet hurt—what do I have?” and it may just give you something right.
Rodney Brooks (Founder and Chief Technology Officer of Heartland Robotics and former Director of the Computer Science and Artificial Intelligence Laboratory at MIT): I think one of the really interesting things here is, because of the way Jeopardy! is set up, it really relies on wordplay and subtlety. It’s forcing the IBM people to take that into account. Now, we use subtleties in our language all the time. We’re just not aware of them because we’re so good at it and so natural.
In one of the examples of Watson playing Jeopardy! was the clue, “A garment a small girl would wear on an operatic ship.” And the answer was “pinafore.” That pulled in so many different pieces of knowledge in just three seconds. What search would you use on Google to get that right now? You’d have to have all the thinking in your head. This is making the computer do it.
Ferrucci: One of the other questions I like is, “If you’re standing, what direction do you look to see your wainscoting?” This sounds like such a simple question, but it’s really fascinating what you have to do to answer that. I mean, humans sort of have to be situated in the world. They have to have some experience in interior design. They have to know what it means to be standing, what relative directions are. It’s just really dramatic to think about what humans do to carefully understand and disambiguate and get at precise answers with language.
IMPLICATIONS FOR US
What if Watson wins? What would that say about us humans?
Brooks: Well, from someone who’s not involved with building this particular machine, whether it wins or not, to me, is irrelevant. It’s being able to be in the game that is the really big advance. Secondly, just because we’ve built something that can do new things that we couldn’t do before doesn’t mean there’s lots of other things the computer can’t do yet. But that it can do these things now, I think will help us tremendously in building search engines and interfaces.
And, you know, Garry Kasparov got beaten by another IBM machine—he was the world chess champion—and people are still playing chess. So I don’t think it will stop us.
Von Ahn: Yeah, I have a very similar answer. I think it would be a tremendous achievement to win, but probably the bigger achievement is the fact that Watson is already competitive against some of the top players in the world, and it’s definitely not going to put us out of a job just yet, but maybe later.
“Sometimes the computer is really sure it knows the answer and wants to be very aggressive with the buzzer. Other times it’s not so sure.”
What do you say to that, Dave? Are you guys building a machine that will take over using intelligence from humans?
Ferrucci: No. I think what this challenge helps us appreciate, frankly, is how incredible the human brain is. And it helps illuminate what’s really hard for computers and what humans find natural and what we’re looking for in terms of the right sort of human-machine interface. Wouldn’t it be great to be able to communicate with the computer like Captain Picard or Captain Kirk does on “Star Trek,” where you can fluently dialogue with an information-seeking computer that can understand what you’re asking, ask follow up questions, and get exactly at the information that you need? That would be incredible. That’s kind of this motivating vision, and whether Watson loses or not in this big game is really not the point. The point is we were able to take a step forward in that direction, and I think that’s what we’re most excited about.
A SELF-CONFIDENT COMPUTER
Playing successfully on Jeopardy! often involves ego and the confidence to gamble. Did you program self-confidence into Watson? How close are computers to breaking that barrier of human-like emotions?
Ferrucci: That’s a great question. It’s a fascinating one really. And one of the big challenges—and this is where we exploited machine learning in a big way—was computing that confidence and figuring out how to use that confidence to manage risk during a game.
So, for example, sometimes the computer is really sure it knows the answer and wants to be very aggressive with the buzzer. Other times it’s not so sure, and it actually weighs how good its competitors are. Other times it feels its way ahead and doesn’t want to take a risk, so it needs to be a lot more confident to buzz in. Sometimes it’s desperate and actually wants to take a risk, even if it’s not as sure. All that’s in there, believe it or not. And you want to call those emotions. They’re really not emotions. They’re complex mathematical equations that we’ve trained into Watson over many, many simulations. It makes it a very fascinating challenge.
The other thing I’ll mention about emotions, though, is Watson doesn’t sweat! I sweat, but Watson doesn’t. So we’ve seen games where Watson lost a big daily double and went down to zero and just kept right on going. Personally I would have fainted. [laughter]
Brooks: Interestingly, when Garry Kasparov was beaten by Deep Blue many years ago, he said, “Well, at least it didn’t enjoy beating me.” [laughter]
So we, as humans, like to hold onto what we’ve still got that the computers don’t have. And what’s happened here is there’s an extra piece. You know, wordplay and puns and stuff like that used to be something that people could have but computers could never understand. Dave and his team now have them understanding it.
Ferrucci: I don’t think you should worry. I mean, think about it this way: A computer is understanding language the way you might understand another language you don’t know. Pick a language you don’t know and then think about it. The only way you could understand it is by reading dictionaries in that language. You don’t really connect those words to your experiences. They’re not connected to your emotions. They’re just connected to one another. The computer is using statistics. It doesn’t actually enjoy or love or appreciate any of what those words represent.
Brooks: Yet. [laughter]
WHERE WE STAND
Yeah, are we on the verge of “Skynet,” the artificially intelligent system in the Terminator movies?
Brooks: We knew Skynet was going to come up! [laughter] You know, one of the things people worry about is, as computers get smarter, are they going to replace us? Well, you are still here. Computers didn’t replace you. You use them as tools. And this is getting better tools. And I suspect that’s IBM’s agenda here—they want to build better tools for people. I don’t think we need to worry anytime soon about the machines taking over. I work in robotics, and the robots we build haven’t gotten rid of people. They just make them more productive. We can relax for a few hundred years, is my guess.
“There are simple things that three-year-olds can do that computers cannot yet do.”
Von Ahn: I think a few hundred years is a good answer. There are the very, very simple things that computers still cannot do. Even determining who somebody is from an image or whether something is a cat or a dog from an image is something that computers cannot do very well. So there are simple things that three-year-olds can do that computers cannot yet do.
So we don’t have to worry about HAL from 2001…
Von Ahn: We don’t have to worry about HAL just yet. Maybe in a hundred years or so. But that’s okay; we’ll be dead. [laughter]
One last question for Dave: What’s the silliest answer you ever heard from Watson?
Ferrucci: Well, as Watson’s developed over the years, it’s had a lot of silly answers; there’s quite a variety of them. I guess one of my favorites is we asked it “What do grasshoppers eat?” and its answer was “kosher.” [laughter]