Anyone who has followed that last few presidential elections has seen how easily data can go from a hero to a zero. Nate Silver was a genius when his model and analysis successfully predicted Obama’s electoral wins with great precision in 2008 and 2012. However, after the 2016 election, Mr. Silver and the value of data fell precipitously from grace in the eyes of political pundits and many voters.
Those of us who know better, well… know better. The problem wasn’t with the data or the models (not primarily at least), it was the analysis of the data that failed. FiveThirtyEight’s final forecast had Clinton at a 71% chance of winning. That is high for sure, but to give some perspective, that equates to about a 6 or 7 point favorite in an NFL football game. No one throws the bookmakers out with the bathwater every time a touchdown favorite loses. This happens nearly every week. In fact, teams favored by 7 – 10 points lose over 25% of the time according to this analysis by WagerMinds.com.
Anyone with a basic understanding of statistics should not have been blindsided by Trumps’ win. Nor should anyone be surprised that so many failed to understand the data in proper context. Las Vegas thrives in part because of peoples’ inability to properly interpret data.
So what lessons can those of us who do not predict football games or elections learn from this? Hopefully, if data professionals learned anything, it is their responsibility to educate the consumers of their data/analysis (executives, clients, etc.). Know that people often hear what they want to hear. As data professionals, it is incumbent on us to be transparent as to the limitations of the data we collect and the models we produce. Draw very clear boundaries around the usefulness of your model. Explain, in practical terms, what your statistic mean (and what they don’t mean). When an executive thinks you’ve “cracked the code” with your model, it will be tempting to ride that wave. Don’t. Being hero for a day runs the risk of over selling and under delivering. This could lead to a lack of trust in you and your data.
Alternatively, while you might curb some enthusiasm initially, educating your data consumers will build trust in you and the data. Educated consumers are more engaged and more satisfied consumers. Consider your success as a data scientist, analyst, or whatever role you play, to be directly tied to the sophistication of your stakeholders as consumers of data.
