How can I get a forecasting model to improve its forecasts over time instead of fitting to training data?

Question

Say I have a model that I'm using to forecast demand for some product. I can train it on some training data in order to get good predictions.

But, the predictions it outputs will only really match past demand. This seems to be a problem related to the data that we give the model because if the past demand data is only how much we sold, but that was our supply as well, then this wouldn't be able to help improve future predictions.

For example, if I had in supply 10 units of product x, and I always sell 10 units of that product, then based on this training data, my model will always say that I should always order 10 units for every day. But, really, it might be better to order 25 units because if I'm always selling everything, then there might be more demand than supply. The problem is that I don't know what the true demand is.

So my question is, how would I get my model to account for that and actually improve predictions rather than trying to fit to historical data.

Answer 1

Like you stated, you get good predictions based on the information in the model.

The model conflates demand and supply in the case of under-supply, since it has no information about unfulfilled demand: how many products were requested but not sold due to lack of supply. Instead it tracks $sold = min(demand,supply)$.

Extend the Model

The obvious solution is to have a separate feature for the true demand rather than the conflating it with unit sold. However, this data may not be available.

Learn from Correlations

Another option is to use richer model: if you have multiple products where most of them don't sell out (i.e. where $sold$ reflects the true demand) and their sales are correlated, then it can predict that a particular product needs to be resupplied at a higher rate than the observed sales. This prediction would be based on the correlations with products that don't hit the $min(demand,supply)$-cap and .

Reinforcement Learning

Finally, another approach is to look to Reinforcement Learning where the agent learns the best resupplying strategy by experimenting.

In a simple setup, your agent will select between two actions: one that orders according to its prediction in the normal case and another action that orders more than predicted. In the case where the prediction matches actual demand the probability of the over-ordering action will be quite low, but it will still be selected with some small probability $\epsilon$.

Reinforcement Learning allows the agent to learn the relative probability of the two, and based on the cases where the actual demand was higher than what you had in stock, it will slowly learn to order more with a higher probability until the prediction based on past sales increases to a point where there is no extra sales to gain from ordering more and the probability of taking the over-ordering action starts to fall again.