This is the next blog in my random series on better understanding some of these advanced Artificial Intelligence and Deep Learning algorithms. This “episode” takes on Generative Adversarial Networks (GANs). Hope you enjoy my “Deep Learning” learning journey.

I originally wrote in “Transforming from Autonomous to Smart: Reinforcement Learning Basics” how Reinforcement Learning was creating learning agents to beat games such as Chess, Go and Mario Bros. Reinforcement learning creates intelligent agents that learn via trial-and-error how to map situations to actions so as to maximize rewards. Reinforcement Learning is one of the more powerful Artificial Intelligence (AI) concepts because it is designed to learn and circumnavigate “situations” where you don’t have data sets with explicit known outcomes (which represents most real-life situations, like operating an autonomous vehicle).

However, creating Reinforcement Learning agents that can win at Chess, Go and Mario Bros is a (relative) piece of cake because each of these games operate within deterministic models where the output of the model is fully determined. We are now seeing Reinforcement Learning agents winning at games that operate under Stochastic models like StarCraft II and Texas Hold’em Poker where the agents need to make decisions and an opponent’s resources are unknown and historical information about previous game play is important to countering bluffing strategies.  All of these advancements are moving us closer to a word of autonomous vehicles and (evil) robots.

Now add the ability of Deep Learning algorithms to create adaptive content based upon the human’s current capabilities and situational status. Using Generative adversarial networks (GANs), analytic-driven environments will be able to create adaptive content that can assist humans in resolving rapidly evolving, difficult situations, or push the envelope in providing a dynamic learning environment that is tailored to each individual’s experiences and strengths. The worlds of education, entertainment and healthcare (not to mention warfare) will be changed forever.  Let’s explore further.

What Are Generative Adversarial Networks (GANs)?

Generative adversarial networks (GANs) are deep neural net architectures comprised of two neural nets, pitting one against the other (“adversarial”):

• The Generator is a neural network trained on a distribution of data (where a distribution of data might be images of dogs, cats, wolves) and extracts features from that distribution of data (mixed with some level of randomness) to try to fool the Discriminator into thinking the fake image is a real image. That is, the generator tries to create a fake cat image based upon the modeling (feature extraction) of real cats to fool the Discriminator into thinking that the fake cat image is a real cat image.  Probably sounds better when someone else explains it…
• The Discriminator is a convolutional neural network that is trained to predict whether input images are real or fake. The discriminator outputs “1” if it thinks the image is real, and “0” if it thinks the image is fake. The discriminator tries to find a “decision boundary” or clusters that separate images into the appropriate classifications (dogs versus cats versus wolves). Then, to classify a new animal as either a dog or a cat or a wolf, it checks on which side of the decision boundary it falls and makes its prediction accordingly.

GANs can learn to mimic any data distribution (diseased plants, cancer, skin lesions, tumors, broken bones, mechanic parts abrasions) to create real-world like experiences. GANs can create something totally new yet life-like based upon adding randomness to a deep understanding of existing data distributions.

How Do Generative Adversarial Networks (GANs) Work?

After reading through 5 steps, see the illustration in Figure 1.

• Step 1: Like a traditional Neural Network, the Generator is trained (extracts features) on a large volume of a particular data distribution (dogs, cats, cars, plant diseases), then adds randomness to produce a fake data distribution based upon the aspects of the real images from which it learned.
• Step 2: The Discriminator then tries to separate the true data distribution from fake data distribution (clustering) and then calculates an error as to how close the fake data distribution is to the real data distribution.
• Step 3:Using Backpropagation, the Discriminative updates weights and biases to the neurons to decrease the classification error.
• Step 4:Then the Generator updates its weights by increasing the randomness on the key features and tries again to fool the Discriminator.
• Step Done:The GAN is considered successful when the generator can effectively trick the discriminator a pre-determined percentage of the time.

Figure 1: “Understanding Generative Adversarial Networks (GANs)”

At each iteration of the training process, the weights and biases of the generative network are updated in order to increase the classification error (error gradient ascent over the generator’s parameters) whereas the weights of the discriminative network are updated so that to decrease this error (error gradient descent over the discriminator’s parameters).

Our good friend Backpropagation (see the blog “What is Explainable AI and Why is it Needed?”) is how the measurement errors are fed back to the GAN to change the neuron weights and biases in order to gradually improve the “realness” accuracy of the fake images (see Figure 2).

Figure 2: Role of Backpropagation in GANs

The result is an entirely new yet real-looking reincarnation of a conglomeration of images for which the GAN has learned all the key aspects of those images and created something entirely new yet very life-like.

Using GANs to Create New Content

GANs are being used to recreate virtual levels and worlds, replicating very closely to how a human would operate.  For example, we’re seeing GANs autonomously creating new levels, like in the game DOOM (DOOM is one of my favorites; hopefully Wolfenstein is next!).

Figure 3: Using GANs to create new Doom Levels

While using GAN to breathe new life into old games is cool, but there are even bigger opportunities for GANs in content generation.

VentureBeat reported that Disney is using GANs to accelerate the creation of storyboard anima…directly from scripts.  Think about the massive volumes of animation images and content buried in the Disney vaults.  Now image how Disney could apply GANs to accelerate the development of new animations more rapidly and at a fraction of the cost of producing an animation today (see Figure 4).

Figure 4: Disney Using GANs to Convert Scripts into Animations

The economics for content companies like Disney could be staggering!  Frozen cost ~$150M to create, 4 years to create, and has netted Disney over $1.2B in box office receipts and another $5B in licensing revenues.  The economics are pretty impressive, and one can certainly see why content providers like Disney would want to accelerate their ability to leverage their wealth of animation data distributions to accelerate these sorts of returns on investments.

Using GANs to Drive Adaptive Learning

While GANs appear to be very useful in creating new real-like content for the gaming, entertainment and advertising industries, the bigger win might be in the ability to create adaptive content that can accelerate training and education. Adaptive content is individualized content that can drive personalized interactions based upon an individual’s capabilities. experience, mood, and immediate goals.

In the area of education, GANs could create dynamic adaptive content that could be personalized for each individual student based upon their experience, areas of expertise, and current areas of struggle.  The GAN-based educational system could dynamically create new content that could help the student in areas in which they are struggling and could push students into new areas more quickly if they are grasping concepts faster. GANs might play an even bigger role in the retraining of our workforces (especially when coupled with immersive technologies like Virtual Reality and Augmented Reality) as technology advances the nature of work.  Learning will no longer be dependent upon the speed of the content and the expertise of the instructor!

And in the world of healthcare, the ability to dynamically create new experiences in training and re-certifying health care providers could ensure that all doctors are able to leverage the last learnings and developments and not be dependent upon having to attend the right conferences (snore) or churn through a stack of research studies to find learnings that are relevant for their current level of expertise.  GANs could also provide a solution to the HIPAA privacy concerns by creating life-like data sets of severe medical situations without actually having to disclose any particular individual’s private data.

And in the area of warfare, well, all sorts of bad things can be conceived with adaptive content that can confuse enemies (and sway political elections).

Generative Adversarial Networks (GANs) Summary

The combination of deep reinforcement learning (to create AI-based assistants) and GANs (to create dynamic, adaptive content) could greatly accelerate the adoption and institutionalization of new technologies and accelerate business and society benefits in the process.  Deep Reinforcement Learning could create AI-based assistants (agents) that aid teachers, physicians, technicians, mechanics, engineers, etc. while GANs could create new adaptive learning environments so that those assistants/agents are constantly learning – learning while the human in the process is doing more human-related activities like engaging with students, patients and customers. 

Blog key points:

• Deep Reinforcement learning is driving rapid advancements in mastering games of increasing sophistication – starting with Chess, Go and Mario Bro with deterministic game frameworks to more stochastic games such as StarCraft II and Texas Hold’em Poker
• Generative adversarial networks (GANs) are deep neural net architectures comprised of two neural nets, pitting one against the other (thus the “adversarial”).
• GANs are playing an increasingly important role in content generation – creating life-like data distributions (cats, dogs, cars) based upon a deep understanding of the features of those data distributions.
• GANs are comprised of two neural networks – a generator which creates the fake images based upon a deep understanding of the features of real images, and a discriminator that tries to detect the fake images from the real images.
• The integration of Reinforcement Learning and GANs could yield dynamic, Adaptive Content that will greatly accelerate the worlds of entertainment, healthcare, education, security and unfortunately, warfare.