Contributed by Ricky Yue. He enrolled in the NYC Data Science Academy 12-week full time Data Science Bootcamp program taking place between July 5th to September 23rd, 2016. This post is based on their second project – R Shiny, due on 4th week of the program. The original article can be found here.
In the last season of the major European soccer leagues, Barcelona, Paris Saint-German, Bayern Munich and Juventus won the title in their own league. The big surprise came from Leicester City, an underdog that was crowned the champion of England. Some would argue that the success of Leicester City well reflects the best part of soccer – its unpredictability. Others appreciate a dominating power like Barcelona which has brought modern soccer to a new level with a delicate balance of individual talents, teamwork and control. If you like watching soccer, what’s your flavor? In other words, do you prefer watching a more predictable match with one dominating side or a less predictable match that is highly competitive? You might have your own preference. This blog introduces an interactive Shiny app that demonstrates the predictability of European soccer matches on three levels: league, season and team.
Data Source
From Kaggle’s recently published European Soccer Database, I used 8 European soccer leagues’ match data from seasons 2008/2009 to 2015/2016. Those leagues are England Premier League, France Ligue 1, Germany Bundesliga, Italy Serie A, Netherlands Eredivisie, Portugal Primeira Liga, Scotland Premier League and Spain La Liga. Variables that are used to examine match predictability include match id, league name, country name, season, home team name, away team name and betting decimal odds given by 10 bookmakers.
Match Predictability and Betting Odds
So how does one measure game predictability? When the referee blows the full time whistle, there are only three outcomes: home win(HW), draw(D) or away win(AW). Each outcome might be accompanied by a range of emotions by the spectator, from disappointment to anger, elation to surprise. Only the bookmakers, tasked with objectively considering the odds, are watching with equanimity. The bookmakers predict the probabilities of all possible outcomes of a match and convert them to betting odds. These odds indeed reflect the objective chance for any of the match outcomes at a particular moment before the end of a match. For instance, if a bookmaker gives a match decimal odds of 1.5, 3.5 and 7.0 for HW,D and AW, the outcome of home win, draw and away win will have a predictive probability of 0.67, 0.29 and 0.15 respectively.
Use Entropy to Measure Predictability
Based on Information Theory, Shannon’s Entropy(H) can be used to measure the uncertainty of an event’s outcome. Given the probabilities of all three outcomes of a match, H can be calculated as H=-(P1*log2P1 + P2*log2P2 + P3*log2P3). H reaches its maximum when all three outcomes have equal probability where H=-(3*(1/3)*log2(1/3))≈1.58. This would indicate that a match is extremely difficult to predict since all three outcomes are equally possible. When the probability of one outcome is very close to 1, let’s say a bookmaker is almost 100% sure Barcelona will win a match, H is approximate to 0. It would suggest a very predictable match. Therefore converting a bookmaker’s betting odds to entropy allows us to measure the uncertainty of a soccer match. For a soccer match with three possible outcomes, the entropy has a range in [0,1.58], with 0 indicating an extremely predictable match and 1.58 indicating an extremely unpredictable match.
Calculate Entropy with Multiple Bookmakers’ Betting Odds
Since one bookmaker tends to attract other bookmakers to advertise similar odds for a certain match, the deviation of odds among bookmakers is pretty small. However, when adding up the probabilities of the three outcomes of a match converted from the betting odds, the sum is usually between 101% and 125%, which is surprisingly not 100%. The portion that exceeds 100% is known as “overround” which represents the bookmaker’s expected profit. In an ideal situation, if a bookmaker accepts $120 in bets at his own quoted odds, he will pay out only $100(including returned stakes) no matter what the actual outcome of the match. Different bookmakers might set different overrounds. The true predictive probabilities of a match from bookmakers can be derived by normalizing the overround such that, P’1 = P1 /(P1 + P2 + P3), P’2 = P2 /(P1 + P2 + P3), P’3 = P3 /(P1 + P2 + P3). The true probabilities are used to calculate the match entropy. The predictability used for analysis equals the average entropy from multiple bookmakers. This metric is supposed to hold more information about the match predictability than any single entropy from one bookmaker.
A Brief Demo of the App
Below is the main interface of the Shiny app. The predictability measured by entropy is shown in the map below. The 8 countries with respective leagues are colored based on the degree of predictability as shown in the tab of “Soccer League Map”. The lower predictability has a darker color and the higher predictability has a lighter color. It appears that France Ligue 1 is the least predictable among the 8 leagues during the 2015/16 season and Spain La Liga is the most predictable.
Moving to the next tab of “Trend by Season”, you can check the change of predictability since the 2008/2009 season for each league. As shown below, it has become more and more difficult to predict the England Premier League since 2013/2014. The past 2015/2016 season is the least predictable season. Indeed, if you’re not familiar with the English league, let me remind you that there was a dark horse named Leicester City that then beat multiple big names including Manchester City, Chelsea and Liverpool. Leicester City went on to win the title with a 10 points’ lead on the final scoreboard.
With a further look into the 2015/2016 English league in the “League by Season” tab, Leicester City has an average entropy of 1.51 in the full season, making it the least predictable team in the league.
Now let us look at the Spanish league shown below, the average predictability has maintained a relatively high level since 2011/2012. In other words, the Spanish league is relatively less competitive. We know that Barcelona has dominated the league in recent years and bookmakers tend to be more optimistic towards their win. Therefore the matches involving Barcelona were relatively easy to predict. Paris Saint-German and Bayern Munich were also dominating their own league respectively. However, those two leagues, especially French league, were much more competitive than the Spanish league. Why is that?
By comparing the Spanish and French league in the two graphs below, the Spanish league actually has two very predictable teams including Barcelona and Real Madrid. Their advantages have strengthened steadily since 2008/2009. In the French league, Paris has shown an obvious lead, and has been more predictable since 2012/2013. Other teams’ predictability are relatively low and distributed in a narrow range. One interesting observation here is that it’s oligopoly instead of monopoly that might level up the predictability of a league. That makes sense in that when two or more super teams are in a league there will be more matches with a dominating team and thus be more predictable.
If you check the table below and rank all Spanish teams by predictability, it’s easy to see that Barcelona and Real Madrid are dominating the league with a large margin of predictability.
Takeaways
After a brief introduction to this Shiny app, you might have found some interesting insights regarding the predictability of European soccer leagues in the past few years. Oligopoly appears to make more league matches less competitive as compared to monopoly. After a season, one underdog might be enough to surprise the whole league.
Next Steps
Given the data, some interesting “next step” questions might be:
- Is there any correlation between the degree of predictability of a team and its final rank after a season?
- Could the app be used to predict the final rank in a league?
- What predictability level would allow a team to win the league title?
- How useful is the monopoly, oligopoly or underdog distinction in helping to make predictions
Now Play It
Now, I guess you might want to play with this app. Please go ahead and click to start.
