I wanted to get your thoughts on a problem I have been facing. I have daily level product sales information (about 4 years). The sales are affected by the typical factors such as seasonality, day of week, "quality", marketing spend (which is sometimes not visible ex-ante- making it a confounding variable, perhaps giving rise to causal inference type approaches OR a latent variable approaches).
The main challenge, however, is the fact that each product only sells for 8-12 weeks (short life cycle product, perishable). That is the entire life cycle of each these products. The problem is to forecast, given a particular day, what is the sales of individual product and then add them all up to find the total sales of the vendor (who sells these products). We know what type of product will be launched but we do not know the exact quality of the product or the marketing spend ex-ante. Other variables such as market potential, day of release, holiday are usually known to us.
Please note the sales of the product is high in weeks 1-2 and then decreases as time goes by ( a decay factor) and not all products are similar, that is, the curves cannot be successfully averaged to produce a representative curve.
Notes: We have tried regression based approaches (which have a high error) as well as the Kaggle favorite XGBoost algorithms (and RF) for the model to learn the non-linearities but the problem is the short-life cycle makes a lot of outlier data points (meaning they are actually not outliers from a business perspective). Open to trying RNN, LSTM etc if they are the right approach.
Thanks in advance!!