Summary: Workforce forecasting and scheduling applications are rapidly upgrading their use of AI. Techniques of time series forecasting ranging from the simple Holt Winters to the complex, DNNs and Multiple Temporal Aggregation are available on some but not all platforms. Increasingly, AI differentiates the usefulness of these apps.
There’s much talk these days about the use of AI in various HR functions from candidate hiring to predicting which employees are most likely to leave. These use techniques like NLP for resume screening or even some light weight classification in the case of churn (assuming you’ve got enough data to train).
But there’s one application in HR that in just the last few years has used good heavy-duty machine learning to solve a serious problem. That’s time series forecasting of needed staffing levels in functions with many workers doing similar jobs.
Call center operations seems to be the one industry most strongly driving sophisticated AI-based forecasting solutions. Clearly there are many others that can benefit ranging from retail to healthcare to any of the many gig-economy business models where too many workers means unnecessary cost and too few means potentially unhappy customers.
It’s likely that call centers led this evolution because the platforms designed to help manage the process automatically collect a huge amount of granular data about demand and worker performance. The fact that this data is already present in electronic form facilitates creating the modeling features for different model types. But platforms that service broad industry types or even other industry specific platforms are rapidly catching up.
Gartner identifies the sophistication of staff scheduling and forecasting techniques as one of the most significant variables currently differentiating these platforms, with only a few offering a full range of forecasting algorithms.
You can find these in the broad-case HCM (human capital management) suites that seek to integrate all aspects of HR, or the more focused WFM (workforce management) applications that typically seek to at least integrate:
- Working time and absences.
- Creating efficient work schedules, some including shift swapping and planned absences, with the ability to ensure that necessary skills are evaluated.
- Ensuring compliance with external rules from federal and state labor laws, local union rules, and any other agreements or local policies regarding working time, pay, or leave.
More Data Makes Better Forecasts
When you’re evaluating platforms or even tasking your data science team to upgrade your forecasts there are several choices and tradeoffs to be considered. And while there are applications targeting companies with as few as 25 to 50 hourly scheduled workers, the real payoff will come when you have many more. Some platforms claim to be able to handle detailed forecasts for as many as 10,000 workers but a sweet spot seems to be in the hundreds to a few thousand. In this range there will also be significant historical data to help make your forecasts more accurate.
How Many Forecasts Will You Need
Chances are you will need several different forecasts with different time frames depending on the business problem you’re addressing. Each of these may be best served by different techniques.
Typically, but depending on your business, you might need:
- Interval forecasts for durations as short as 15 to 60 minutes over a single day.
- Daily forecasts.
- Weekly forecasts.
- Annual forecasts.
While all forecasts need to separately consider level, trend, and seasonality, the shortest term forecasts, interval and daily forecasts must also consider special events or times of day which can be anticipated but don’t follow a smooth pattern. These might arise from:
- Changes in a marketing promotion
- A new product introduction
- Weather factors
- Special events (e.g. Super Bowl or flu outbreaks)
- Equipment failures
These need to be understood and isolated in the historical data before putting them through your forecasting routine and anticipated as new events in the forward forecast.
What Types of Time Series Forecasting Work Best for Workforce Forecasting
When evaluating forecasting applications or working with different techniques, these four are currently most common though not all are available in all platforms.
Holt Winters (Triple Exponential Smoothing)
Holt Winters has been the go-to technique for some years. The reference to ‘triple’ smoothing is its ability to separately consider level (last month’s actuals), trend (short or long term growth), and seasonality (variations over a year based on seasonal factors) in a single calculation.
The advantage is that this can be performed on a spreadsheet. The disadvantage is that it is easy to over fit the data by selecting the wrong smoothing coefficient for each factor. Also the resulting forecast is as the name implies ‘smoothed’ so that short term granular trends critical to short term forecasts may be lost.
ARIMA (Auto Regressive Integrated Moving Average) and ARIMAX
ARIMA has proven accurate in many complex situations and requires a professional level understanding to select the right version. There are several including for example Double Seasonal ARIMA developed at Oxford University that allows you to layer in multiple seasonalities.
For example you could set one seasonality to account for 30 minute intervals (48 time periods in 24 hours) and a second to weekly trends (336 time periods of 30 minutes).
As this illustration shows, ARMIA and its variants are able to produce sophisticated forecasts taking multiple patterns into account. 
Image source: Rob Hyndman
Neural Networks
It’s no secret that over the last few years multi-layered deep neural nets (DNNs) have been used to discover and predict complex patterns. Time series forecasting shares elements of the two most common applications of DNNs, that is CNNs (convolutional neural nets) typically used for image classification (considered static data even if video) and RNNs (recurrent neural nets and its several variants) used for text and speech recognition and response (considered time series data).
The topic is too long for a deep dive here except to say that CNNs have also been used successfully in time series forecasting and have a speed advantage because they can be deployed in massive parallel processing (MPP) which RNNs cannot. Another factor is how much data you have as both types require large amounts of training data.
In short, DNNs can be an effective time series forecasting technique but require deep data science skills to execute.
Multiple Temporal Aggregation (MTA) / Holistic Integrated Forecasting
At the leading edge of time series forecasting are evolving techniques that combine elements of different methods in a kind of ensemble method that allows simultaneous consideration of long and short term trends and even special events.
Consider for example this time series forecast impacted by seasonality, special events, weather, and underlying trend. The forward forecast is quite complex and takes all these elements into consideration.
Image source: SAS
This work is based on an R routine called MAPA – Multiple Aggregation Prediction Algorithm, which produces some promising forecasts. There is also another MTA algorithm called Thief. Some of the major analytic platforms like SAS have similar approaches.
Final Considerations
The economics of your business and the costs and benefits of avoiding over-staffing and under-staffing will guide how much effort you are willing to put into achieving increasingly accurate and complex forecasts.
As you move from Holt Winters to ARIMA to Neural Nets and finally to Multiple Temporal Aggregation the need for specialized data science skills and understanding increases dramatically. Simpler may be good enough but increasingly WFM platforms will differentiate on their ability to provide these increasingly complex and accurate forecasts.
Other articles by Bill Vorhies
About the author: Bill is Contributing Editor for Data Science Central. Bill is also President & Chief Data Scientist at Data-Magnum and has practiced as a data scientist since 2001. His articles have been read more than 2.1 million times.
He can be reached at:
