Home > Unstructured Thoughts > Text Analytics & Jeopardy

Text Analytics & Jeopardy

February 14 – 16 2011, IBM is making a big gamble by attempting to compete against Ken Jennings and Brad Rutter on Jeopardy

Named Watson after one of the founders of IBM, DeepQA uses the OASIS text analytics standard UIMA on an asynchronous messaging infrastructure (presumably MQSeries) for real-time analysis of unstructured content (a question) and response (answer).

As pointed out by Stephen Baker in an article on Huffington Post this isn’t a “search” problem that any Tom, Dick or Google can solve. 

Take for example the question “Not surprisingly, more undergrads at the University of Puget Sound are from this state than any other.”  Watson needs to understand that “not surprisingly” means it’s true, or likely and also be able to reason from there what state the University of Puget Sound is in and whether Washington is the most likely true answer.

Watson is to be fully self contained with no access to external information – just like a real contestant. 

If IBM can pull this off, it will show that “search” isn’t “text analytics.”  That’s not to say that text analytics can’t (and doesn’t) deliver value to search technologies, but to try and meet the Jeopardy challenge with ancient Bayesian algorithms or traditional search indexing simply can’t work.

Dave Ferrucci is the Principal Investigator leading the Watson team, and was recently interviewed on NPR.  You can read the transcript here:  http://www.npr.org/2011/01/08/132769575/Can-A-Computer-Become-A-Jeopardy-Champ

13-Jan-2011 update:

There was a test match today that Watson won.  You can see photo’s from the match on cnet News:  http://news.cnet.com/2300-11386_3-10006289.html?tag=mncol

Advertisements
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: