Predictive analysis is a journey of refinement of data over time using a predictive model. Right from choosing the right model to refine your data to the amount of time and effort invested, there are many thing which may go wrong in your way.
Predictive analytics is also the enabler of Big Data; businesses collect vast amounts of real-time customer data and predictive analytics uses this historical data, combined with customer insight, to predict future events. However, it all goes back to the use of the Legacy Data coupled with the right kind of predictive model.
Below are five most common mistakes businesses(after investing huge amounts in setting up data science department) should keep in mind while working with predictive analytics.
1. Correlation does not mean causation: Oldest rule in the book. While creating correlation matrix the most common mistake done is considering that if there exists correlation between two variables, it implies that one causes the other. Not always true.
2. Not Looking for patterns: This holds true in most of the big data analysis for ex. decision trees can grow exponentially large, so be on the lookout for patterns when you use them. If you’re taking a test and you forget some rules of combinatorics, making a simpler instance of your test question may help you find patterns more quickly. Models capture relationships among many factors(& variables) to allow assessment of risk or potential associated with a particular set of conditions, thus decision making. This also brings us to the third point,
3. Failing to consider enough variables: When deciding which variable should be used to include in a model, one should include every variable that seems predictive, most of the data models remove unnecessary variables/factors while computing. Also lack of creating and including custom variables (Using the by operator ex. Visitors by visit) is another mistake one should avoid.
4. Using Future Data to create Future Data: After generating a desired(as close as possible) result using one technique, most of the times the data is not saved or is ignored; The magic lies in saving the result and cross-computing it again with the current data or previously saved results. Ex. In Data Workbenchone can save the obtained results after analysis of a dimension with the other to create a new dimension for the future use. Not using this technique is a big miss!
5. Lack of Experimentation: Most of the times when a company invests in good analytical tool it is trying to obtain results as did another; even the data models used are ones which have proven success with a competitor. There is nothing wrong with it but there is no guarantee that a predictive model which brought 90% success rate for one company, will also get same or better for the other. One point commonly skipped while selecting models is the Data, Your company’s data might be very different from the other even being a competitor, the anomaly does not lies in the type of data but in the behavior of data. The solution is to not just adapt a model but to tweak it in a way which is feasible for your data.
This post was first published on LinkedIn
