Logistics Demand Forecasting with 20k Different Time Series

Question

I'm trying to tackle a very challenging problem and I would appreciate your help.

My organization has a lot of different items which can be demanded by our clients. Those items can also be returned back to our warehouses. Our goal is to predict how many instances of each item our clients (which are actual warehouses) will demand next year.

As for now, my organization is predicting the demand using a weighted average, based on the quarterly demands of the last 7 years. This gives us a prediction for the demand for the next quarter. Then, they multiply this prediction by 4 to get the next year's prediction. This is my baseline to beat. Our data is irregularly sampled, meaning that demand can occur any day. This also means that we have to resample the data ourselves (we tried monthly and quarterly).

Each demand has a few features:

1. Item Id
2. Date of Demand
3. Receiving Warehouse - our organization has a lot of warehouses spread out across the country, this is the warehouse that received the item.
4. Sender Warehouse - the warehouse that sent out the item.
5. Type of Demand
6. Quantity - can be either positive (if the client consumed the item from the warehouse) or negative (if the client returned the item back to the warehouse)

Again, for clarification: the goal is to predict the Total Quantity of the next year for each item. Obviously, this is a time-series forecasting situation. The problem is that we have over 20K different time series (because each item’s history is a distinct time series).

What we have done so far:

1. Simple statistic models (exponential smoothing, Holt-Trend Methods, etc.)
2. Classical machine learning models (linear regression, decision trees, etc.)
3. Ensemble methods (random forest, boosting, etc.)
4. Simple deep learning models – neural networks with a few linear layers and an RNN layer.
5. Complex state of the art neural networks for time series forecasting (we tried Google's Temporal Fusion Transformer)
6. Calculating the regular weighted average (as the company does now) but only after removing anomaly demands (based on a DBSCAN algorithm).

The main problem is that none of the above methods can beat the regular weighted average for all the time series we have – meaning that for many of them the new proposed model preformed even worse than the weighted average. Also, we couldn’t find any pattern that allowed us to match each time series to a "winning" method.

What we are looking for is a method which will perform better for all the different time series.

Any help will be highly appreciated.

Thanks in advance.

Answer 1

You could also try ARIMA & SARIMA.

ARIMA has three hyper parameters (p,d,q) & SARIMA (seasonal ARIMA) has additional three parameters (P,D,Q). You could find the best hyperparameters for your dataset using grid search.

https://people.duke.edu/~rnau/411arim.htm

Answer 2

First of all, there is no guarantee that you can beat a baseline with machine learning. This is something you should take into account.

You might also find a solution for some particular Type Of Demand due to some inner behavior, but this might not generalize well for the rest of the cases.

I would also try to use Facebook`s Prophet

From the authors:

Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. It works best with time series that have strong seasonal effects and several seasons of historical data. Prophet is robust to missing data and shifts in the trend, and typically handles outliers well.

It might work well out of the box without any major tweaks. You can also run it in parallel (because the process of fitting the model is kind of automatic), which is helpful when you have this much time series.

Here is a link on how to implement it efficiently (but note that original docs are quite good as-well).

And a post regarding parallel execution.

I would also spend more time analyzing the results: can you check the error between different types of demand, search for the particular cases when prophet beats baseline, and try to understand why it is so and so on.

Answer 3

You've tried a lot of modelling options, among them several that tend to work in the setting you describe. If none of this was better than a moving average, it's likely that modelling is not the bottleneck of your problem. To find out what it is, play around with these questions:

• Have you done enough exploratory analysis and talking to business experts to understand the data and the data-generating process?
• What kind of performance evaluation strategy are you using to evaluate your models? There's more choice than most people realize, and there's no generically best option. If you're wondering what I'm talking about, this paper might give you some fresh ideas.
• What kind of features might be well-suited to aid the predictions?
• What happens when I feed a complex model (complex = more complex than a running average) with simple data, i.e., only the past periods you feed into your running average. Use these and only these as an input to LightGBM and see what happens. Look at the first trees that LightGBM fits, and where it goes wrong. If it doesn't go wrong, feed additional data features into it until it does.
• You get the idea and you'll probably have some other ideas of your own. Just don't try the next modelling technique. Look at everything else.