Demand Forecasting for Multiple Products Across Thousands of Stores

Question

I'm currently working on a demand forecasting task, with data on tens of thousands of products across a couple thousand stores. More specifically,I have 3 years' worth of daily sales data per product in each store, and my goal is to forecast the future sales of each item in each store, one day ahead; then two days ahead, etc.

So far I've considered breaking down each product-store pair into a single time series, and doing a forecast for each time series as was done in Neal Wagner's paper, Intelligent techniques for forecasting multiple time series in real-world systems. In other words, I will use only the historical information of a particular store's sales of the product to forecast the future sales of that product in that store.

However, I've been browsing Kaggle and competitions like Corporación Favorita Grocery Sales Forecasting suggest a different approach, which is to use the information from all stores and all products to predict future sales. As I understand it, historical sales information of all products in all stores are dumped into the training set, from which the model will learn to forecasts future sales. It's very different from traditional time series methods, but apparently, based on the results of the competition, it works.

The latter method seems promising and more robust. However, there's the problem of having to process hundreds of millions of data points.

Which method is more appropriate for my task? For those who have worked on similar problems, which methodology would you recommend?

Answer 1

I was also in a similiar situation. I had a dataset with number of stores and product categories and I needed to predict weekly sales for each category. You may want to check gradient boosting alghorithm and random forest which give promising results in terms of forecast accuracy. Also, regardless of the method, the best thing to do is to create dummy variables as creating single time series is very time consuming.