Summary: IBM’s Watson as it exists today is as close as we’ve come to a single integrated platform for AI. It contains all the capabilities for image and video, natural language speech and text input and output, and the most comprehensive knowledge recovery module yet combined together. If you want to exploit the advances we’ve made in AI you need to understand where Watson is today and where it’s heading.
Recently we wrote about how the ‘popular’ Watson of Jeopardy fame still lingers in the memories of our non-data scientist colleagues and perhaps misleads them about the capabilities of AI. But that’s the Watson of 2011, a virtual eon ago in DS time, especially with the leaps-and-bounds progress in AI. It’s time we got in tune with the modern Watson, or more correctly IBM’s Watson Group and its Watson platform and took a look at all there is to offer.
Three Capabilities of AI (Cognitive Computing)
First let’s review. There are three broad capabilities in today’s AI and they are:
Image and video processing: Largely driven by Convolutional Neural Nets (CNNs) this field has been getting most of the press with capabilities like facial and object recognition.
Text and speech processing: This is generally known as Natural Language Processing (NLP), the capability to not only intake speech and text but to understand the nuanced context of strings of words, to perform search or translation on them, and output words and blocks of text and speech. Siri, Alexa, and Cortana represent examples of this field that used to be the butt of jokes but are now more than commercially acceptable. This field has been largely driven by Recurrent Neural Nets (RNNs).
Knowledge Retrieval: This field is a combination of search and Question Answering Machines (QAMs like Watson). It’s almost not fair to put these in the same category since the standards for search are only to return a list of possible sources in which the answer may be found. A QAM is required to return the single most likely answer while interpreting the context and nuance of the speech or text input. Watson at its historical core is a QAM and this is still its strong suit.
Search can rely on a variety of older techniques associated with much more simple routines. For the most part these don’t qualify for what we think of as AI but they’re included here since they are a major form of output from our virtual AI assistants like SIRI. Still there is a crossover area between simple search (what year was George Washington born) versus the much more complex capabilities of QAMs (given these symptoms, identify the best course of medical treatment).
Knowledge Retrieval QAMs are Quite Complex
Watson and QAMs in general have not received nearly as much press as the other two capabilities which really fails to recognize the sophistication of what’s going on here. QAMs rely on three capabilities:
- Natural language processing to understand the complexities of human communication both as a means of user input, as the basis for its more sophisticated search, and of course as output.
- Hypothesis generation and evaluation. QAMs create many hypotheses from the input language then search the knowledge base to find potential matches for each. By applying advanced analytics to weight and evaluate a number of potential responses based on only relevant evidence they arrive at a single conclusion.
- Evidence-based learning based on outcomes to get smarter with each iteration and interaction.
Combining these three areas, image processing, text and speech processing, and knowledge retrieval we get the eyes, ears, mouth, and memory content that represent what AI is today. Given this anthropomorphic description, it’s no wonder that many people believe a more accurate descriptor is ‘Cognitive Computing’.
The reason for this lengthy preamble is that while all three of these major component areas have been moving forward at a rapid pace, we haven’t till now seen the introduction of a comprehensive AI platform where the best of all of these can be found in a single place. There is an arms race going on here and all the majors are moving to become the ‘one place’ for AI. In fact, in an effort to win over developers and grow their ecosystems, as of just the last year much of this capability is now open source.
Watson as Comprehensive AI Platform
By early 2014 IBM had seen the writing on the wall and had established The Watson Group as a separate business within IBM. IBM CEO Virginia Rometty says this is a major battleground that IBM must win.
Moving quickly, IBM’s Watson Group led with a series of products to appeal to its enterprise partners and by mid-2015 had added a very large list of cloud based APIs and developer SDKs to encourage the largest possible group of outside developers.
David Schubmehl, an analyst at IDC compares IBMs new playbook in AI with Microsoft’s Windows in personal computing and Google’s Android OS in mobile. “IBM is trying to do the same thing with Watson,” he said, “open up a platform, make it available for others, and democratize the technology.”
Here’s What You Can Find in Watson Today
Products for Enterprise End Users
Watson Virtual Agent
This bot is intended to supplement or replace many of your CSRs and perhaps even with better quality responses and higher customer satisfaction. It’s not limited to CSR activities but could be used in any customer interactive service and has text or conversational speech inputs and outputs. There are onboard analytics that help to analyze trends in customer concerns and evaluate satisfaction with the interaction. IBM has even pre-trained several packages for specific industry use so the need for additional customization is limited.
Watson Explorer
Not ready to have your customers handled directly by AI? Explorer is intended to quickly bring together a 360 degree view of the customer or prospect plus recommended solutions based on its analytics. CSRs who spend much time trying to access information, or particularly sales and marketing staff interacting with customers are targets for this more internally focused product. Financial services, investment advisors, and insurance companies are good examples of the target market.
Watson Analytics
This is IBMs shot at Citizen Data Scientists and offers automated data prep, modeling, and data visualization. From my quick tour it looks like the modeling is limited to which variables are most likely to affect outcome but that’s a good place for analysts to start. There’s a free version and two expanded versions with greater input capabilities and these premium versions also include Twitter data.
Watson Analytics will also suggest questions that might be answered by the data you load so this is a guided process. There’s a separate version called Watson Analytics for Social Media that provides access to a much wider range of social media input. This won’t produce sophisticated predictive models that can be implemented in operating systems but it will let individual users explore statistical inference and sentiment analysis from social media in an extremely easy and non-technical format.
Watson Knowledge Studio
This is the enterprise in-house version of the developer APIs and SDKs below that allow significant customization of the Watson enterprise products.
Watson Customized for Specific Industries or Processes
There are a large number of offerings that combine elements of Virtual Agent, Explorer, and Analytics into packages targeting specific industries or processes:
- Commerce
- Education
- Financial Services
- Health
- IoT
- Marketing
- Supply Chain
- Talent
- Work
Watson Health
This is actually a separate business unit from the Watson Group formed specifically to exploit Watson in healthcare. This is a cloud based service that provides a secure and HIPAA compliant platform. IBM has pre-established compliance with medical compliance and regulatory issues allowing users to utilize the service without this infrastructure burden. Much of the press here has been focused on drug research and the ability of the Watson QAM to surface new relationships among drugs, molecules, or even DNA that suggest routes to new treatments.
For Developers
Having strong in-house products is one thing but to win this market space IBM knows it must enlist as many developers as possible to their platform. To that end they have expended great effort to make all these components available to developers via API.
As of September 2015, about the same time the Watson Group opened its Silicon Valley location, they already claimed 350 company partners using Watson to make products, with about 50 services on the market and some 70,000 software developers using Watson software in some way. In this last year that number is sure to have increased.
Here’s the list of current developer packages including some that are still experimental. For a full description of what each of these contains try this IBM page.
- Alchemy Language
- Conversation
- Dialog
- Document Conversion
- Language Translator
- Natural Language Classifier
- Personality Insights
- Retrieve and Rank
- Tone Analyzer
- Speech to Text
- Text to Speech
- Visual Recognition
- Alchemy Data News
- Discovery
- Tradeoff Analytics
- Project Intu
Is Watson Actually a Comprehensive AI Platform?
Watson is not a platform for creating new data science. It is a platform for exploiting the latest advances in the data science that underlie AI. In the examples above you can see that it covers all the big 3 areas of AI, image processing, text and speech processing, and knowledge retrieval. It does not replace analytic platforms for predictive analytics but it can give the citizen data scientist a leg up in structured and unstructured data analysis. Most importantly though, it is a platform for the development of applications that commercially exploit how far we have come in AI.
About the author: Bill Vorhies is Editorial Director for Data Science Central and has practiced as a data scientist and commercial predictive modeler since 2001. He can be reached at:
