I am trying to create a model to predict the units that will be sold for different grocery items say in the next week.

I am structuring the problem in a three-step procedure.

  1. Group together the selling data from a macro-category (e.g. if I want to predict "egg box XYZ" I will consider all the items from category "eggs") summing the selling data from all items of that category for each day. Then I use prophet to predict the selling data for next week for the whole category. I am doing this step as the "whole category" data have more history and should be more robust compared to single items data.
  2. Implement a prophet model on the specific items. I expect the error to be higher since the data are less consistent.
  3. Implement a regressor (could be GradientBoost, RandomForest etc.) on the single item using as input variable for a specific day: the previous week selling data for that item, the forecasted sold units for the whole category, the forecasted units (again with prophet) for the specific item, whether we are in a particular time of the year (e.g. boxing day), promotions applied etc. The idea behind this regressor would be to dampen the error of the second step. However I cannot lower the error from the second step.

How could I approach this problem?


There is an ongoing competition on Kaggle right now which is about sales prediction. https://www.kaggle.com/c/m5-forecasting-accuracy/notebooks

The user kyakovlev have created a series of notebooks where he creates features and creates an lgbm model:


The notebook linked is the last in the series. I've had success in applying a very similar strategy on a real-world project for a client.

You probably want to engineer more features and add more lagging variables.

The prophet model could possibly be a problem if the patterns in the data aren't clear. I have had examples where it is completely of. This could be due to the trend. The Prophet model also easily overfit on holidays due to the lack of data. This is due to the nature of the model. You should definitely inspect what the model is learning.

Models such as XGBoost or LightGBM are great for this type of problem!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.