When you give customers advice that can help them save some money, they will pay you back with loyalty, which is priceless. Interesting fact: Fareboom users started spending twice as much time per session within a month of the release of an airfare price forecasting feature. This tool continues to grow conversion for our partner.
Besides travel, price predictions find their application in various scenarios. Commodity traders, investors, construction developers, or energy generators use estimates on future price movements for business purposes.
This time we talked with experts from AleaSoft, ENFOR, REALas, and our own data science specialist to answer the question: How to implement price forecasts on markets with high volatility? The article describes the steps to build a price prediction solution and implementation examples in four industries.
What is price forecasting and how is it done
Price forecasting is predicting a commodity/product/service price by evaluating various factors like its characteristics, demand, seasonal trends, other commodities’ prices (i.e. fuel), offers from numerous suppliers, etc.
Price forecasting may be a feature of consumer-facing travel apps, such as Trainline or Hopper, used to increase customer loyalty and engagement. At the same time, other businesses may also use information about future prices. Entrepreneurs may need to define an optimal time to buy a commodity to adjust prices of products or services that require a commodity (lumber, coffee, gold), or evaluate the investment appeal of fixed assets.
Price prediction can be formulated as a regression task. Regression analysis is a statistical technique used to estimate the relationship between a dependent/target variable (electricity price, flight fare, property price, etc.) and single or multiple independent (interdependent) variables AKA predictors that impact the target variable. Regression analysis also lets researchers determine how much these predictors influence a target variable. In regression, a target variable is always numeric.
In general, price forecasting is done by the means of descriptive and predictive analytics.
Descriptive analytics. Descriptive analytics rely on statistical methods that include data collection, analysis, interpretation, and presentation of findings. Descriptive analytics allow for transforming raw observations into knowledge one can understand and share. In short, this analytics type helps to answer the question of what happened?
Predictive analytics. Predictive analytics is about analyzing current and historical data to forecast the probability of future events, outcomes, or values in the context of price predictions. Predictive analytics requires numerous statistical techniques, such as data mining (identification of patterns in data) and machine learning.
The goal of machine learning is to build systems capable of finding patterns in data, learning from it without human intervention and explicit reprogramming. To solve the price prediction problem, data scientists first must understand what data to use to train machine learning models, and that’s exactly why descriptive analytics is needed.
To learn more about a machine learning project structure, check out our dedicated article
Then the specialists collect, select, prepare, preprocess, and transform this data. Once this stage is completed, the specialists start building predictive models. A model that forecasts prices with the highest accuracy rate will be chosen to power a system or an application. So, the framework of the price prediction task may look like this:
- Problem statement.
- Understanding of market peculiarities. Answering the question: What factors impact prices of a commodity/product/service?
- Data collection, preparation, and preprocessing.
- Modeling and testing.
- Deployment of a model into a software system or application.
Now that we know a typical price prediction project roadmap, let’s explore real-world examples from the energy sector, travel and hospitality industry, and real estate.
Electricity price forecasting: the combination of statistical and machine learning techniques
By the early 1990s, the energy sectors in many countries were fully regulated and monopolized. Government agencies and local bodies were monitoring the work of utility companies, setting their terms of service, pricing, construction plans, ensuring these companies adhered to safety and environmental standards.
Then a shift towards deregulation began, the main goal of which was to reduce electricity costs and ensure a reliable supply of energy via competition. The power industry started turning into a free market where prices for products and services depend on supply and demand. In other words, the market players trade electricity on exchanges like other commodities. The participants set their bids and offers while trying to maximize their profits. Deregulation is an ongoing process across markets.
Factors affecting electricity demand and price: weather changes, transmission, regulators, fossil fuel prices, and others
Electricity is a special commodity type, so trading it is a tricky task. It’s non-storable (must be supplied immediately once generated/must be generated and used simultaneously), so a balance between production (generation) and consumption (load) is crucial for energy system stability.
The demand for electricity and, consequently price, depends on the weather (temperature, precipitation, wind power, etc.) and changes in daily and business activities (weekends and weekdays, on-peak and off-peak hours). Non-storability of electrical energy and continuous shifts in demand lead to electricity price volatility.
Continue reading
