1
$\begingroup$

I wanted to get your thoughts on a problem I have been facing. I have daily level product sales information (about 4 years). The sales are affected by the typical factors such as seasonality, day of week, "quality", marketing spend (which is sometimes not visible ex-ante- making it a confounding variable, perhaps giving rise to causal inference type approaches OR a latent variable approaches).

The main challenge, however, is the fact that each product only sells for 8-12 weeks (short life cycle product, perishable). That is the entire life cycle of each these products. The problem is to forecast, given a particular day, what is the sales of individual product and then add them all up to find the total sales of the vendor (who sells these products). We know what type of product will be launched but we do not know the exact quality of the product or the marketing spend ex-ante. Other variables such as market potential, day of release, holiday are usually known to us.

Please note the sales of the product is high in weeks 1-2 and then decreases as time goes by ( a decay factor) and not all products are similar, that is, the curves cannot be successfully averaged to produce a representative curve.

Notes: We have tried regression based approaches (which have a high error) as well as the Kaggle favorite XGBoost algorithms (and RF) for the model to learn the non-linearities but the problem is the short-life cycle makes a lot of outlier data points (meaning they are actually not outliers from a business perspective). Open to trying RNN, LSTM etc if they are the right approach.

Thanks in advance!!

$\endgroup$
  • $\begingroup$ How are you validating your model? You state that you add each individual product per vendor to get the total sales of the vendor. Does this mean that your end goal is to predict total sales per vendor (as in, this is the final target variable that is used in your loss function, like root mean squared error?). Furthermore, you state products only sell for 8-12 weeks. If this is a consistent pattern, then surely a "week" time variable will model this fairly easily? Like, how I see it; a regression tree learner will learn that after say, >9 weeks the product will have decreased sales at... $\endgroup$ – aranglol May 30 '19 at 4:22
  • $\begingroup$ a greater rate? Also, you state that "not all curves cannot be successfully averaged to product a representative curve". Are you implying then that you are creating a single model per product? Or are you creating one giant model, trained on multiple time series expanded over many rows, as is the typical approach when using machine learning based models for time series? If you are doing the latter, then what causes these differences between the curves? I hypothesize that this is what needs to be modeled in some fashion, like product category maybe (food? clothing?) Usage type? etc. $\endgroup$ – aranglol May 30 '19 at 4:25
  • $\begingroup$ Personally, I have a hard time believing that all your products are distinctly unique such that a model cannot learn general patterns that all the time series exhibit. If this is the case, however, then have you tried univariate time series methods like ARIMA/SARIMA/ETS/TBATS? You could use hierarchical forecasting methods; basically fit many individual time series methods at the product level, and then use least squares to force the individual time series to add up to the aggregated (total per vendor) one. See otexts.com/fpp2/hts.html $\endgroup$ – aranglol May 30 '19 at 4:35
1
$\begingroup$

So essentially, you have products i, each with a short time dimension t. Not clear what this is, but if you (as it appears) pool over the entire time (and thereby products), you mix up different products i. If there is any information on how the quality (determining demand) of each product i is regarded, you may differentiate by quality of product i and then look at how demand (controlled by quality) changes over time (in which the product is offered). So imo you should not stack products over time, but see if you can explain how demand for each single product is determined. In essence you have many short timeseries. You should look at what the drivers in the level of sales between these timeseries are.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.