0
$\begingroup$

I'm trying to tackle a very challenging problem and I would appreciate your help.

My organization has a lot of different items which can be demanded by our clients. Those items can also be returned back to our warehouses. Our goal is to predict how many instances of each item our clients (which are actual warehouses) will demand next year.

As for now, my organization is predicting the demand using a weighted average, based on the quarterly demands of the last 7 years. This gives us a prediction for the demand for the next quarter. Then, they multiply this prediction by 4 to get the next year's prediction. This is my baseline to beat. Our data is irregularly sampled, meaning that demand can occur any day. This also means that we have to resample the data ourselves (we tried monthly and quarterly).

Each demand has a few features:

  1. Item Id
  2. Date of Demand
  3. Receiving Warehouse - our organization has a lot of warehouses spread out across the country, this is the warehouse that received the item.
  4. Sender Warehouse - the warehouse that sent out the item.
  5. Type of Demand
  6. Quantity - can be either positive (if the client consumed the item from the warehouse) or negative (if the client returned the item back to the warehouse)

Again, for clarification: the goal is to predict the Total Quantity of the next year for each item. Obviously, this is a time-series forecasting situation. The problem is that we have over 20K different time series (because each item’s history is a distinct time series).

What we have done so far:

  1. Simple statistic models (exponential smoothing, Holt-Trend Methods, etc.)
  2. Classical machine learning models (linear regression, decision trees, etc.)
  3. Ensemble methods (random forest, boosting, etc.)
  4. Simple deep learning models – neural networks with a few linear layers and an RNN layer.
  5. Complex state of the art neural networks for time series forecasting (we tried Google's Temporal Fusion Transformer)
  6. Calculating the regular weighted average (as the company does now) but only after removing anomaly demands (based on a DBSCAN algorithm).

The main problem is that none of the above methods can beat the regular weighted average for all the time series we have – meaning that for many of them the new proposed model preformed even worse than the weighted average. Also, we couldn’t find any pattern that allowed us to match each time series to a "winning" method.

What we are looking for is a method which will perform better for all the different time series.

Any help will be highly appreciated.

Thanks in advance.

$\endgroup$
0
$\begingroup$

First of all, there is no guarantee that you can beat baseline with ML, this is something you should take into account. You also might find a solution for some particular Type Of Demand due to some inner behaviour, but this might not generalize well for the rest of the cases.

I would also try to use Facebook`s Prophet

From the authors: Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. It works best with time series that have strong seasonal effects and several seasons of historical data. Prophet is robust to missing data and shifts in the trend, and typically handles outliers well.

It might work well out of the box, without any major tweaks. You can also run in in parallel (because the process of fitting of model is kinda automatic), which is helpful when you have this much time series.

Here is a link on how to implement it efficiently (but note that original docs are quite good as-well) https://towardsdatascience.com/implementing-facebook-prophet-efficiently-c241305405a3

And a post regarding parallel execution

https://medium.com/spikelab/forecasting-multiples-time-series-using-prophet-in-parallel-2515abd1a245

I would also spend more time analyzing results: can you check the error between different types of demand, search for the particular cases when prophet beats baseline and try to understand why its so and so on.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.