Summary: Proof of Concept projects are a popular place to start but they may be the wrong solution. To ensure success focus on Proof of Value and alignment with the company’s strategy. Get the right executive sponsor and keep them involved.
If you Google ‘Data Science Proof of Concept’ you will find dozens if not hundreds of articles extolling the virtues of starting with a POC. And yet it is a common experience that while a POC may lead to a larger implementation of what the POC was designed to demonstrate, this method of getting started with predictive analytics and Big Data is frequently not in the interest of the company as a whole.
What Exactly is a POC
Here’s what we mean by a POC.
- This is most typically a project in the $30K to $50K range representing about 3 to 6 manweeks of data science labor by a team of two or three spread out over no more than 4 or 5 calendar weeks.
- The client contributes a minimal amount of labor in project management and coordination, some SME time, and a minimal amount of technical labor in extracting whatever current data is being used.
- The client makes no investment in hardware, software, or skills in their IT or business departments. These projects are most frequently performed off site in the platform developer’s or consultant’s data lab.
- At the conclusion there is a demonstration of success against the objective, however, implementation of the model or the Big Data architecture is almost always outside the scope of the POC.
In short, these are extremely narrowly defined demonstrations that leave little or no value behind except for having provided the sponsor with cover against risk and now the ability to go and ask for funding for this specific application.
The typical sponsor of a Big Data and predictive analytics POC is a senior or mid-level manager in IT or on the business side, but not C-level.
What’s wrong with this? If you’re a Big Data or analytics platform developer probably nothing. The successful POC allows you to sell your platform to the client – mission accomplished. If you’re a data science consultant, same outcome. You sold some hours and hopefully established the basis for more billable work in the future.
But if you’re the company you got:
- A demonstration project proving feasibility but leaving little or nothingt behind in terms of data-driven operational capability.
- Nothing that brought the company as a whole closer to a culture of data-driven thinking.
Why Have So Few SMBs Embraced Predictive Analytics and Big Data?
There are still no unbiased surveys of actual penetration of predictive analytics and Big Data among SMBs but it’s estimated to be suprisingly low. About two years ago Gartner estimated it was no more than about 12%. Given the wide press coverage we might generously estimate 20% today. What’s going on?
One of the chief culprits is the ongoing belief among C-level execs that Big Data and predictive analytics is just technology. If it’s just technology then we’ll wait for the IT department to explore it and they’ll let us know how it should be fit into our existing technology stack.
Why Don’t POCs Drive Company-wide Change?
John Weathington wrote a good article a few years back in which he observed:
“As I scan the most prominent literature on leading change from legendary authorities like Kurt Lewin and John P. Kotter, nowhere do I find the idea of a “proof of concept.” In fact, the only place I come across this construct is in IT circles. That’s because IT people come from the engineering subculture, and in their culture you never release anything that’s not perfect. The proof-of-concept is a way for them to reduce risk with experimentation without releasing a sub-par product.”
Real executive leadership is missing in the POC driven model that treats Big Data and predictive analytics as just technology. The CIO budget holder makes the decisions about allocating a (small) portion of his budget to this fancy new technology.
The POC-driven model combs through some portion of the company’s data looking for something, anything that works. When a POC is successful the CIO can present findings to the business leads and perhaps then those business unit heads for whom the POC has some benefit will provide some sponsorship.
It’s not unusual for this model to continue for some time. The result can be some data science capabilities but ones that are separate and siloed and treated as narrowly defined tools. These are typically not efforts that have been intentionally aligned with the company’s strategic objectives.
If Not POC, Where Should You Start? With Executive Sponsorship and a Data Strategy.
If you do a quick read through of some of the Gartner or O’Reilly studies you’ll quickly see that a lack of executive sponsorship is one of the major barriers to adoption. So isn’t the POC a good way to get the attention of the C-level? Yes and no.
If as we described above it leads to the adoption of a series of stand alone ‘technology projects’, then no. If it was really necessary to start with little firecracker POCs to demonstrate the explosive strategic value of becoming data-driven, then maybe so.
Here’s a simple change of mindset (borrowed from John Weathington referenced above) that instead of focusing on Proof of Concept, we should instead create projects to demonstrate Proof of Value. By focusing on value we change the orientation so that any projects are aligned with value to the company. In other words, they are aligned with the company’s strategic objectives.
So What’s a Mid-Level Project Sponsor To Do?
Let’s suppose you are the same mid-level manager, either IT or business side, who believes that predictive analytics and Big Data have value for your specific area of focus. In the past you are exactly the person who would try to find a small amount of budget and design a POC. If the project is successful you hope it will win you a bigger budget for implementation of your specific project.
Let’s take an example. You are a product manager in a national retail chain with hundreds of stores and an on-line catalogue sales channel as well. You’re responsible for selecting and pricing product in your particular line and making sure the inventory for each store and the on-line channels is lean but never out of stock. POS data from the stores comes through a third party service and is delayed by about a week. The on-line sales data is in a separate system and a slightly different format. It takes about two weeks to get the files you need to analyze and much longer than that if anything changes and IT has to make any modifications to the reports and files you receive.
You know this is a perfect problem to be solved by a simple NoSQL database and some blending and advanced analytic tools. With all the data in the same place, as soon as it happens, and with the right tools for forcasting volumes, you could even do more advanced things like dynamic pricing. This looks like a POC to you.
Here’s what we suggest.
- Think like an executive and reframe the problem as a Proof of Value project aligned with a larger strategic goal of the company. In this case you might frame this as dramatically increasing the speed at which the company can react to change and make more profitable decisions. A recent report by New Vantage Partners says that 83.5% of the 50 fortune 1000 companies interviewed agreed that speed, in terms of the time it takes to get answers, make decisions and go to market, is a key value of Big Data and predictive analaytics.
- Get an Executive Sponsor, preferably the CEO or General Manager. Having the proper level of executive sponsorship removes the risk of failing to recognize the value of your project to the entire company. The executive sponsor will gurantee that the project goals reflect value to the company as a whole and can be more widely diseminated after your success. This approach also keeps the executive sponsor engaged and avoids miscommunication about the opportunity. (Note: If your company already has a Chief Data Officer or Chief Analytics Officer start there.)
- Your POV is now the first small win. Yes you still need a modest and lean project to start with which will be the seed on which bigger projects utilizing the lessons learned can be rolled out.
Some Tips for Your POV Project
- Don’t shoot for perfection. Take a lesson from the startup world and make the MVP (minimum viable product) your goal. The approach is lean and conservative. With an executive as sponsor chances are your budget will be greater than your initially envisioned POC.
- Do think about how the POV will be integrated into actual operations. This requires thinking as much about process as it does thinking about technology. This means involving users beyond your immediately defined goal so that you’ll be gaining support and additional cheer leaders. At the end of your POV you’ll have a roadmap to real value.
- Don’t worry too much about selecting a specific Big Data platform or predictive analytic toolset. If you need a Big Data platform you’ll be better off going with a cloud product like AWS with no need for immediate investment. The same is true of the predictive analytic tools. This is probably a good time to bring in a consulting data scientist who will bring their own tools, once again avoiding time and cost spent on platform selection. The point of the POV is as much about lessons learned as about success. When you’re finished you’ll have a much better understanding of the skills and infrastructure required.
- Be prepared to pivot. Even though you may not have experience with advanced analytics and Big Data there is very little risk of absolute failure. Almost any decent body of data will yield some valuable signal to improve operations. Once again a good outside data science consultant can help you avoid common mistakes. It’s an experiment. Be ready to pivot when evidence leads you in a direction you didn’t at first consider.
- Be modest in your goals. If you consult with others regarding what the POV should be, avoid the most sophisticated and complex applications in favor of something simple and straightforward. A few predictive models can yield impressive value. If someone suggests you shoot for say optimization, that can be quite complex and should not be your starting point.
- Avoid the bells and whistles at first. Usability is just as important as feasibility. For example, if your project may result in automatic notification of required actions based on your analytics, consider using manual distribution of the findings before you go all in for automation. Think of this as the conciege model of implementation which later may be automated.
The key is for your project to be linked to the company’s strategy. Then think big, start small, and build on the momentum of the first small wins to grow your data-driven culture based on advanced analytics.
About the author: Bill Vorhies is Editorial Director for Data Science Central and has practiced as a data scientist and commercial predictive modeler since 2001. He can be reached at:
