Architectural Primer: Conversation as a Platform (CAAP) on Azure

According to Mckinsey report, Artificial Intelligence (AI) is poised to unleash digital disruption and companies needs to start preparing for this new wave now. In 2016, companies invested $26 billion to $36 billion in AI technologies. Virtual agents a.k.a chatbots are an important component of AI technologies and we are seeing more and more interest among our customers to integrate chatbots into their business process. However, chatbots are not silver-bullet for all problems and Jarvis is still a far-fetched mythical personification apt in the world of Avengers.

In this article, I will introduce one of the many Architectural pattern for developing a chatbot based on my experience in the field. I will start by explaining the chatbot context and then put across a logical architecture pattern that can be used for developing chatbots.

Let us first discuss taxonomies of models that are prevalent in Conversation as a Platform (CAAP).

Taxonomy of Models:

Virtual Agents, generally, have two taxonomy models:

Retrieval-based models use a repository of predefined responses and some kind of heuristic to pick an appropriate response based on the input and context. The heuristic could be as simple as a rule-based expression match, or as complex as an ensemble of Machine Learning classifiers. These systems don’t generate any new text, they just pick a response from a fixed set.

Generative models don’t rely on pre-defined responses. They generate new responses from scratch. Generative models are typically based on Machine Translation techniques, but instead of translating from one language to another, we “translate” from an input to an output (response).

Chatbot Conversational Framework:

CAAP relies on specific conversational context. These contexts can be classified as follows:

Open Domain: Here the context of the conversation is not defined. There is no well-defined goal or intention. An example of this can be a conversation with Google Home. You ask it whatever you want, it will give try to find answer and provide you a response.

Closed Domain: Most of the chatbot applications fall into this criteria. Each chatbot has a specific context and specialises to answer questions to that domain. Technical Customer Support or Shopping Assistants are examples of closed domain problems.

Logical Architecture for CAAP:

Now that we have a context, lets discuss the logical architecture of how we have implemented closed domain heuristic based chatbots on Microsoft Azure.

Following are the key components of the logical architecture:

• Bot Framework: Microsoft Azure provides Azure Bot Service that is powered by Microsoft Bot Framework and Azure functions that helps to create a serverless bot that scales on-demand. The Bot Framework is a platform for building, connecting, testing, and deploying powerful and intelligent bots. The Bot Framework has two key components:
• Bot Builder: A rich and full-featured SDKs for the .NET and Node.js platforms. The SDKs provide features that make interactions between bots and users much simpler. It is suggested to use a “waterfall” dialog to manage conversation flow in the program.
• Bot Connector: Used to publish the bots created to multiple external channels including email, Facebook, Skype, Slack, and SMS.
• NLP Engine: Natural Language Processing (NLP) is an integral part of developing heuristic rule-based chatbots. It decodes human-machine interaction. In the context of chatbots, NLP classifies the intent of the chat conversation and once the intent is identified, it routes the flow to appropriate dialogue handlers. Language Understanding Intelligent Service (LUIS) is part of Azure cognitive service that abstracts complex NLP models and helps developers to create apps that identifies the correct intent and entities. LUIS also provides GUI based interface for assigning standard responses to intents, training and retraining the intents with utterances etc.
• Response Repository: Response repository is where responses based on identified intents and entities are stored. These responses are stored in a NOSQL database like MongoDB or Azure Document DB.
• Conversation Repository: The bot-human conversation can be stored in a conversational repository. This repository can be used to analyse interaction patterns and corresponding sentiment of the conversation. Although not a mandatory component, this becomes important especially when we want to create bot personality and fine-tune the responses based on the feedback received from the conversation.
• State Repository: Bots have amnesia. One of the challenges faced by developers is to maintain the context of the conversation. For it to remember the context of the conversation, it is important to have a state repository that helps to maintain the context of the conversation for extended period of time. Bot Framework provides dialog contexts that maintains the stack of dialogs that are active in an conversation. In our implementations, we used cache mechanism (Redis Cache) for maintaining longer contexts.

NLP-Bot Builder Interaction:

The flowchart shows the interaction between Bot Builder component and NLP component. As shown, NLP is just one (although important) part of the Bot Architecture. It takes much more than an NLP engine to build smart bots.

Conclusion:

This Architecture is only one flavour of many patterns that can be implemented. Of course, this can be enhanced to connect to internal data sources for logging and authentication etc.

It is the age of AI. Virtual Agents will be ubiquitous and penetrate into every aspect of business processes. Looking forward for the time when Jarvis becomes a reality.