# Strategies for continuously assessing and improving model performance

I am building a supervised machine learning model to generate forecast.

So I would have historic data like this:

SKU, Month, .... other features, Actual Volume


That I can use a model to generate forecast, using the actual volume as the label.

Of course, there would be a variance between the forecast volume and the actual volume

What are the proper ways to leverage such data, without generating any data leakage, to incorporate such info to train the model to minimize the variance?

Should the data be fed back to the data with moving average, etc. and retrain, or is there other better strategy to properly assess the performance of the model and learn from it?

The data will be time series data with various features such as exchange rate, salesperson, etc.

• I may misunderstand, but are you saying that you want to include the prior month's forecast error into the prediction for the current month's volume? This seems like a bad idea, as the forecast residuals should be random noise. If there is information in the residuals, then you should be taking that information into account in your original model. Jan 9 '19 at 18:04
• Thank you for your insight. I know ideally the original model will take care of the residual, but I am not 100% convinced that the residuals will always be random noise. That's why I want to have a process in place to track the performance of the model over time to hopefully discover ways to continuously improve the model. Also, I want to observe the bias of the prediction, and sometimes, some systems would provide their own ML algorithm and predictions. We will build our model, but we might also include predictions from other systems and hopefully those will help improve the algorithm too. Jan 10 '19 at 16:40