I love the simplicity of the data science concepts as taught by the book “Moneyball.” Everyone wants to jump right into the real meaty, highly technical data science books. But I recommend to my students to start with the book “Moneyball.” The book does a great job of making the power of data science come to life (and the movie doesn’t count, as my wife saw it and “Brad Pitt is so cute!” was her only takeaway…ugh). One of my favorite lessons out of the book is the definition of data science:
Data Science is about identifying those variables and metrics that might be better predictors of performance
This straightforward definition sets the stage for defining the roles and responsibilities of the business stakeholders and the data science team:
- Business stakeholders are responsible for identifying (brainstorming) those variables and metrics that might be better predictors of performance, and
- The Data Science team is responsible for quantifying which variables and metrics actually are better predictors of performance
This approach takes advantage of what the business stakeholders know best – which is the business. And this approach takes advantage of what the data science team knows best – which is data transformation, data enrichment, data exploration and analytic modeling. The perfect data science team!
Note: the word “might” is probably the most important word in the data science definition. Business stakeholders must feel comfortable brainstorming different variables and metrics that might be better predictors of performance without feeling like their ideas will be judged. Some of our best ideas come from people whose voices typically don’t get heard. Our Big Data Vision process considers all ideas to be worthy of consideration. If you do not embrace that concept, then you risk constraining the creative thinking of the business stakeholders, or worse, miss out on surfacing potentially valuable data insight.
This blog serves to expand on the approach that the data science team uses to identify (and quantify) which variables and metrics are better predictors of performance. Let me walk through an example.
We recently had an engagement with a financial services organization where we were asked to predict customer attritors; that is, to identify which customers were at-risk of ending their relationship with the organization. As we typically do in a Big Data Vision Workshop, we held facilitated brainstorming sessions with the business stakeholders to identify those variables and metrics that might be better predictors of performance (see Figure 1).
Figure 1: Brainstorming the Variables and Metrics that Might Be Better Predictors
Note: I had to blur the exact metrics that we identified for client reasons of competitive advantage. Yea, I like that!
From this list of variables and metrics, the data science team sought to create an “Attrition Score” that can be used to identify (or score) at-risk customers. The data science team embraced the iterative, “fail fast / learn faster” process in testing different combinations of variables and metrics. The data science team tested different data enrichment and transformation techniques and different analytic algorithms with different combinations of the variables and metrics to see which combinations of variables yielded the best results (see Figure 2).
Figure 2: Exploring Different Combinations of Variables and Metrics
The challenge for the data science team is to not settle on the first model that “works.” The data science teams needs to constantly push the envelope and as a result, fail enough in their testing of different combinations of variables to feel personally confident in the results of the final model.
After much testing and failing – and testing and failing – and testing and failing, the data science team came up with an “Attrition Score” model that had failed enough times for them to feel confident about its results (see Figure 3).
Figure 3: Identifying Variables and Metrics that ARE Better Predictors
We needed an approach that got the best out of everyone on the project – the business stakeholders brainstorming variables and metrics, and the data science team creatively testing different combinations. The final results in this engagement were quite impressive (see Figure 4):
- The Dell EMC process led to a model that identified ~59% of attritors
- As a benchmark, American Express publicizes a successful attrition model that identifies 24% of attritors
Figure 4: Final Attrition Model Results
The creative data science process of combining different variables and metrics is highly dependent upon the success of the business stakeholder brainstorming exercises. If the business stakeholders are not brought into this process early and allowed to think creatively about what variables and metrics might be better predictors of performance, then the collection of variables and metrics that the data science team will seek to test will be limited. Put another way, the success of the data science process and the creation of the actionable score is highly dependent upon the creative involvement of the business stakeholders at the beginning of the process.
