Target corporation’s massively profitable data science project threw them into the news spotlight a few years back. Their story makes for a valuable case study in bridging data science and business intuition.
After having painstakingly developed a ‘golden-goose’ analytic model that could flag pregnant shoppers based on seemingly normal purchase patterns, Target began mailing brochures for baby clothing to women who had done nothing to directly indicate to Target that they were pregnant. Target knew that the spending patterns for a pregnant woman were about to change big time, and Target badly wanted that spending to be done in their stores.
The amazing accuracy of these mailings soon came to light when a man famously intercepted Target’s mailing to his teenaged daughter. Target had realized the teenager was pregnant before her own father had.
Target got a lot of bad press for that trick, but we in the analytics world were pretty impressed. Target was also writing a chapter in the playbook for how to bridge data science research and business intuition.
It started with intuition
It’s crucial to realize what launched the data science effort in this case. It started from intuition: A realization about the power of habit. Target realized that the key to winning customers from competitors such as Walmart (the market leader) was to reach out to them at key habit-forming moments.
From this foundational intuition sprang the goal of identifying customers who were approaching life-changing moments at which new habits would form. Further intuition suggested that the most profitable such moment would be the birth of a child. In this way, intuition had formed a key new business goal: identifying the pregnant shoppers and becoming their supplier of choice for baby items.
But how could they identify the expecting mothers and market to them at precisely the right times during the pregnancy?
Enter the data scientists
Target’s data scientists were then given a clear business goal: use your data and models to tell us which shoppers are pregnant. Do this, and we’ll secure loyal customers who will spend heaps of money over the next few years (you parents know exactly what I’m talking about).
This was a few years before Big Data systems appeared, so Target analysts had only traditional data sources at their disposal: point of sale, birth registry, demographic repositories, etc. What the analysts did have, however, were this clear purpose and well-defined goals; goals that were rooted in business intuition.
What the analysts did have, however, were this clear purpose and well-defined goals; goals that were rooted in business intuition.
It was this business intuition that started the team of analysts working for several months on analytic models that would eventually single out the pregnant shoppers with remarkable precision, even identifying their due dates to within a small window (Aside: this was eventually accomplished by focusing on purchases of 25 specific but otherwise innocuous items which had high signaling potential when purchased at certain intervals)
Target made a huge profit from their analytics. The executives had an intuition for strategic growth, and the analysts made it real by collecting data and building accurate models, an effort spanning several months. The result, although questionable in terms of privacy, resulted in a large profit boost that extended over several years.
Data Beats Opinion
But business intuition is sometimes completely and utterly wrong.
Recall Tesco’s seasoned chairman, Ian MacLaurin, who famously told his data consultants, ‘You know more about my customers in three months than I know in 30 years
‘You know more about my customers in three months than I know in 30 years’
and whose management would later say that data “has guided almost all of the key business decisions the management team have made in recent times, reducing the risk of taking bold new initiatives… what we have to do becomes as plain as the nose on your face.” (2)
Or think of the Chicago theatre executives whose misconceptions of theatre guest personas were shattered when an external consultant used customer data to show that guests were coming for completely different reasons from what the executives had assumed, despite their many years of industry experience. (3)
Even business experts need to regularly validate their intuition with data.
A Cycle of Continuous Improvement
Business intuition and data science should together walk through a series of insight-analysis-value cycles.
To illustrate, some time back I was working with a company that had been thrown into crisis by …. you guessed it… a sudden drop in revenue. Sitting in a war room with the company’s senior leaders, one after the other attributed the revenue drop to various intuitive, albeit speculative causes. We only identified the true cause of the revenue drop after several days of digging into the data. Once the analytics had put the company on the right scent, one of the executives most familiar with business trends realized that the same situation would re-occur within 6-9 months. We then refocused our analysis on this future period to forecast financial impact and to identify potentially mitigating steps.
It was another case of business intuition and data insights sitting at the same table and together producing maximum value for the company.
Putting the pieces together
In a previous posting, 150 data scientists and still no business value?, I mentioned how critical it is for data scientists to live and breath business intuition. In the examples above, we briefly illustrate the back-and-forth: how intuition directs analytics while analytics, in turn, guides intuition.
Gartner has developed a very useful four stage value progression, the Gartner Analytics Maturity Model. In terms of this progression, Gartner’s analytic stages one and two (description and diagnosis) serve to validate and inspire business intuition, while stages three and four (prediction and prescription) provide the means to make business intuition operational. It’s a back-and-forth all the way down the value chain.
To quote Simon Uwins: (2)
“We don’t forget our intuition, but better data lead to better thinking, and our data give us the confidence to ask the right questions”
Let’s keep asking the right questions, looking to both data as well as to intuition for the answers.
I’d love to hear peoples’ thoughts and experiences in the comments below.
