TL;DR
Artificial Intelligence, Machine Learning, and Deep Learning models have demonstrated significant power to grow and improve businesses. We have found that the best approach to AI production is what venture capitalists do when they evaluate and invest in startups.
A 2019 McKinsey survey found that a majority of executives whose companies have adopted AI solutions report that these solutions have provided an uptick in revenue, and 44 percent say that AI has reduced costs. However, AI models are often costly to develop and there’s a risk that the model will fail to have a meaningful impact on the organization and languish in pilot purgatory after completion. So how can we maximize the probability of success in such a scenario? How can we create a useful (and used) tool quickly and cost-effectively? We have worked on numerous such projects for a range of organizations. Here is something we learned: the best approach for AI production is similar to what venture capitalists (VC’s) do when they evaluate and invest in startups.
Where to begin? How to Brainstorm AI/ML ideas
It’s important to develop an intuition of what success looks like in any field. Jiro Ono, perhaps the most respected sushi chef in Japan, has a mantra: “In order to make delicious food, you must eat delicious food.” The same is true for AI and ML projects. Immerse yourself in the research and find out what is possible in, for example, natural language processing, computer vision, and object detection. Find examples of successful use cases and learn what you can about them. The Conference on Neural Information Processing Systems (NeurIPS) is a good place to start, as it provides a comprehensive overview of what is being discussed and developed in the field. Transfer research into your domain. Follow the news for your industry and find out what’s trending.
Start with simplicity. Look for manual, mundane tasks that can be accomplished with a few seconds of human thought. These are the top candidates for automation with AI/ML, especially when labeled training data is available for the activity.
Inspire yourself. Look at open datasets. Look at your datasets. What else can you collect? As a brainstorming technique, consider how you would remake your competitors’ organization or the market in general with ML automation (or augmentation)… What kind of insights would that allow you to make?
Before collecting this data look into the possibility of buying it.
Once you have collected several ideas, how can you choose the best ones for investment? There is another group of folks that face a similar scenario of developing many projects and choosing among the best of them — venture capitalists. Let’s see what we can learn from them.
What Can We Learn from VC’s?
First, gather a lot of ideas. Then validate the already promising ones, but also try those which contradict intuition.
Don’t invest a lot until you validate the top risks. Build a Proof of Concept (PoC) of the deep learning network or AI model. Build a PoC of the interface and test it with users.
Pay attention to traction. What is traction for the AI project? You need to look for a group of organization employees who desperately need your solution. Are the employees excited or scared about the prospect of working with an AI or ML model, or of becoming a data-driven organization? If they are excited, they understand that implementation of the model is going to simplify their workflow and allow them to focus on more complex decisions. Buy-in from them is going to be very important for successful implementation.
If the organization’s teams are afraid of the AI/ML model development, provide education and training. A McKinsey survey of “AI high performer” companies found that they make a conscious effort to educate, help, and if necessary, retrain employees to foster a partnership between humans and machines, instead of a replacement or a pilot purgatory scenario. Be careful not to only look at the excitement of your researchers, as most of them are excited by hard problems to solve, not with business impact.
Many startups succeed – some wildly so – but unfortunately most startups will fail. VC’s understand this risk. That is why VC firms build a diversified portfolio. You can follow this lesson by building a portfolio of ideas. Be an explorer of ideas, and be cautious not to favor a single idea at the expense of others. Treat sunk costs as an investment in exploration rather than as a justification for fixating on a single path. Machine Learning projects are research. Be ready for failures. Try different things. Adapt. Pull more resources when investment is less risky, and pull the plug on lines of attack that are clearly failing.
Make Small investments in Development and Construction of AI/ML
Focus on time to value (TtV), which is the time between a business request and the initial delivery of that request. The goal of any business is to minimize TtV in order to realize some level of business value as quickly as possible.
Typically the largest cost is the cost of staff time (i.e. engineers and researchers). If you want to get a good Return on Investment, work on reducing this. Use technology that helps to build stuff faster. For example, in many cases you don’t need to build ML models from scratch like many teams tend to. You can build your AI solution using fitting, pre-made building blocks or at least use high-level tools that get good results quickly.
1) AI products specifically made for the problem you’re solving
2) Existing APIs, e.g. Voice recognition or Object detection API from Google, Microsoft or AWS
3) If that is not enough, consider using AutoML tools to build your custom models
4) If a truly custom model is needed, build them using high-level ML libraries like Fast.ai or Keras and not pure Tensorflow.
Follow the tools space (or partner with someone who does) as it’s changing every quarter.
Use R/Shiny to build PoC of decision support systems. The only reason not to use Shiny is when you have something very basic to test (use BI) or your team is committed to Python and Python only (use Dash). For years now Shiny proved to be superior in this space (Decision Support Systems PoCs).
Reduce infrastructure work. In most cases this is going to be just a fraction of the cost of your project. Help yourself with the right tools:
-
- RStudio Connect
- Domino Data Lab
- W&B
- Neptune.ai
- Google Cloud AI Notebooks,
- AWS SageMaker
Use clean data. No shortcuts here. Invest in building proper data gathering and data pipeline, with a focus on good data quality. Validate and make sure you know what you have. A modest investment in this step will go a long way to ensure good results in all of the following steps.
Denouement: Determine a Clear Cut Off and Listen to users.
Determine a Clear Cut Off. Without the proper guide rails, research can become a black hole. Make sure that people working on the AI/ML models understand their runway and success criteria. Setting this success criteria is difficult, but ultimately necessary. It might require education on both sides; business often asks for 100% accuracy, tech often prefers not to promise anything. Those topics can only be addressed with openness, education, and deep conversations.
Speak with Users early and often. The Y Combinator mantra is “Make something people want.” If the end goal of your work is to make something that your teams are going to use, and I venture to guess that for >99% of team leaders it is, make sure you speak with the potential users as soon as possible. That’s one of the best ways to not only reduce the overall investment but also to make sure you’re building the right thing. An honest conversation with users is worth its weight in gold.
Conclusion
Your chances of success depend a lot on your planning and gathering before you sit down to write model specifications for your engineering team. Here I recommend thinking like a venture capitalist as an approach to growing your org with ML/AI. Gather a lot of ideas. Validate the promising ones, but don’t ignore the counter-intuitive ones. Don’t over-spend, and there are techniques and tools that will help you with this. Build a PoC and test it with users early and often. Use clean data and determine a clear cut-off for your project. And once again, speak with users early and often. Put it together, and you will clear the way to cost-effective, org-changing ML/AI success.
Resources
Time-saving AI infrastructure tools:
RStudio ConnectDomino Data Lab
Money-saving AutoML solutions:
https://cloud.google.com/automl
Shiny (https://shiny.rstudio.com) is an R package that makes it easy to build interactive web apps straight from R. You can host standalone apps on a webpage or embed them in R Markdown documents or build dashboards. You can also extend your Shiny apps with CSS themes, htmlwidgets, and JavaScript actions
