Nothing strikes terror into the hearts of humans more than the idea of an intelligent robot gone bad.  The fear is that a robot can acquire the ability to learn and adapt to the point of superseding their human creators…and with evil intentions.

From Gort (“The Day the Earth Stood Still”) to Sonny (“I, Robot”), films provide a wide variety of potential robot scenarios.  Only a few of these film robots have demonstrated artificial intelligence to the point where they have threatened humankind (like the Cyberdyne Systems series T-800 Model 101 in “Terminator” and the unnamed NS-5 robots in “I, Robot” ).

However the one evil robot that demonstrated its ability to continuously learn through experimentation and failure would be the Omnidroid from the “The Incredibles”.  As representative of the data science process in Figure 1, the development of the Omnidroid occurred over many iterations, with each failure a chance to learn and perfect its killing skills.

Figure 1: Data Science Engagement Process

As we were warned early in the movie by the character Mirage: “It’s a learning robot. Every minute you spend fighting it only increases its knowledge of how to beat you.”  The end result was a seemingly undefeatable robot, the Omnidroid V10 (see Figure 2).

Figure 2: The Omnidroid v10

As we kickoff 2019, a year which promises to be full of simultaneous Artificial Intelligence-induced exuberance and terror, let’s see what we can learn about the future of “smart” products from the Omnidroid’s evolution; that is, how do we create products that increase their knowledge, insights and predictive capabilities through every interactions in order to better defeat you…er, um…serve you.

3 Levels of Advanced Analytics

I wrote a blog a while back titled “Artificial Intelligence is not ‘Fake’ Intelligence” where I introduced the 3 levels of Advanced Analytics because I needed a simple positioning of the advanced technologies like machine learning, deep learning and reinforcement learning for my class at the University of San Francisco.  Let’s apply our learnings from that blog to see how one would apply the 3 levels of advanced analytics to create “smart”, autonomous products, using the development of the Omnidroid as our measuring stick.

Level 1: Insights and Foresight

Level 1 is about codifying cause-and-effect and measuring goodness of fit

Level 1 forms the foundational level that seeks to quantify cause-and-effect, establish confidence levels, measure goodness of fit, and predict what is likely to happen next at some measurable level of confidence.  Level 1 includes the following analytic capabilities:

• Statistics, which is a branch of mathematics dealing with the collection, analysis, interpretation, presentation and organization of data. Statistical methods are used to support hypotheses (decisions) and provide credibility to modeling results and outcomes (via confidence levels and “goodness of fit” measures).
• Data Mining, which is the analysis of large data sets to uncover previously unknown patterns such as groupings of similar activities (cluster analysis), unusual events (anomaly detection), and dependencies between entities (association rule mining, sequential pattern mining).

The Omnidroid results from Level 1 Advanced Analytics? “You clinking clanking clattering collection of caliginous junk!” to quote the Wizard from the “The Wizard of Oz” (see Figure 3).

Figure 3:  Level 1: Insights and Foresight for the Omnidroid

The early Omnidroid versions weren’t successful, though they did have victories over some of the lesser “Supers” (Universal Man, Psycwave, Everseer, Macroburst, Phylange, Blazestone) that inspired further investment. Time to move to the next level of advanced analytics.

Level 2: Optimized Machine Decision-Making

Level 2 is about codifying trends, patterns and relationships buried in the data in order to predict what’s likely to happen next

Level 2 seeks to enable computers to learn on their own; to identify patterns, trends, associations and relationships in data, build models that explain the data, and predict outcomes without having pre-programmed rules and analytic models. Level 2 includes the following advanced analytic capabilities:

• Supervised Machine Learning“learns” of relationships (known unknowns) buried in the data given a labelled result or outcome (fraud, attrition, failure, web click); it uses different Supervised Machine Learning algorithms to “learn” the relations between multiple input variables and an output variable (labelled result) using 1) Regression-based methods to predict outputs based on input variables and uses regression algorithms (e.g., Linear, Logistic, Polynomial, Ridge, and Lasso Regression) and 2) Classification methods to identify to which category a set of classifications belong using classification algorithms (e.g., Support Vector Machines, Decision Trees, Random Forest, and Nearest Neighbor).
• Unsupervised Machine Learningdoes not have labelled outcomes, consequently it uses multiple input variables to discover the previously unknown relationships between those variables (unknown unknowns). A common Unsupervised Machine Learning algorithm example is clustering analysis where the machine finds those clusters after the analysis.  Other Unsupervised Machine Learning algorithms include: association models/market basket analysis, K-means, Principal Component Analysis (PCA), Birch, Ward and Independent Component Analysis (ICA).
• Deep Learningleverages a system of highly interconnected analytic layers to decompose complex data formats (e.g., images, audio, video) in order to learn about the data and create inferences about the data. Deep Learning or Neural Network algorithms include Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Generative Adversarial Neural Network (GANN) and Markov Chain.  Check out the blog “The Mostly Complete Chart of Neural Networks, Explained” for a most excellent compilation of the different Neural Network algorithms.

By the way, the lines that separate these analytic categories tends to blur. For example, is a Markov Chain or a Support Vector Machine classified as Unsupervised Machine Learning or Deep Learning? To be honest, algorithms like these probably fall into both classifications.

And what are the Omnidroid results from the Level 2 Advanced Analytics (see Figure 4)?

Figure 4: Level 2: Optimized Machine Decision-Making

The second generation Omnidroids were successful at amassing insights about the individual Supers in order to defeat them (Gazerbeam, Stormicide, Gamma Jack).  Unfortunately, they had reached the limits of what they could accomplish with Level 2 analytics (which was confirmed by its ass-kicking at the hands of Mr. Incredible).  Time to move onto the Level 3 of the advanced analytics.

Level 3: The Learning and Intelligent Enterprise

Level 3 creates interactive environment that uses trial-and-error in order to maximize rewards while minimizing costs

Level 3 focuses on advanced analytic algorithms that can self-monitor, self-diagnose, self-adjust/fix/heal and self-learn. These analytics perceive the world around them (to assess and quantify rewards and costs), create goals, make decisions towards those goals, measure decision effectiveness, and learn in order to refine the decisions that advance towards the goals.

• Reinforcement Learningfocuses on how software agents take actions in an environment so as to maximize cumulative rewards while minimizing costs. Reinforcement learning uses trial-and-error to map situations to actions so as to maximize rewards. Actions may affect immediate rewards but also affect subsequent or longer-term rewards, so the full extent of rewards must be considered when evaluating the reinforcement learning effectiveness. Examples of Reinforcement Learning algorithms include Q-Learning and SARSA.

And what are the Omnidroid development results from this third generation of Omnidroids (see Figure 5)?

Figure 5:  Level 3: The Learning and Intelligent Product

The Level 3 Omnidroid v10 ultimately suffered the same fact as the earlier Omnidroid versions – defeat.  Some of the failings that lead to the failure of the Omnidroid v10 include:

• No experience fighting multiple Supers simultaneously. It didn’t know how to respond to multiple Supers, each with a different super power, with a coordinated attack pattern (helps the Elastic Girl, Mr. Incredible and Frozone had worked together in the past).
• No experience fighting anywhere but on its native island. It was unfamiliar fighting in a large metropolitan area which offered different fighting resources and hiding places than its native island.

But maybe the biggest downfall for the Omnidroid v10 was the improper assessment of its analytic model “success”.  The Omnidroid v10 thought it had defeated Mr. Incredible (which it hadn’t), which lead to the pre-mature launch of the product which ultimately lead to its failure.  For the analytic models that powered the Omnidroid v10, “good enough” wasn’t actually “good enough”.

Creating Smart Products Summary

Hopefully this was a fun way to look at what is going to be required from an analytics perspective to create “smart” products.  There are many valuable lessons that data scientists can learn from the movie “Mr. Incredible” about the challenges of creating smart products like autonomous vehicles, trains and factory robots.

And maybe the biggest challenge for the development of smart, autonomous products is knowing when “good enough” is actually “good enough”.  When trying to optimize the operations of these smart, autonomous products, one must be prepared to realize that the current path to performance optimization may not actually be the optimal path, and the data science team must be prepared to jettison their existing work and be willing to try a different path that might lead to a better performing analytic model (see Figure 6).

Figure 6: Saddle Points and challenge optimizing across multiple dimensions

This is an important lesson for the creation of our AI-induced “smart” products – that there must be constant testing, learning, and maybe even some unlearning and re-starting afresh in order to find the optimal models.  But Syndrome, the arch villain and the creator of the Omnidroids, will never know if he was on the right optimization path, because he died at the end of the movie. Oh well, end of lesson for him.