How best to explain Forecasts when model cannot be accessed?

Question

My company purchases demand forecasts from an external vendor (after providing them with our historical data). My manager wants to explain the forecasts that we are receiving and has requested for feature importance for the same. The model is a black box model and we are not sure what model it exactly is (The accuracy is pretty good though). We don't have access to the model. I have access to the training data and the prediction data.

How do I approach forecast explainability when I cannot access the model? What other factor should I keep in consideration?

Answer 1

Are you being billed by the number of predictions?

If not, in your batch you can add 1 or subtract 1 (or change the label, etc.) to each field, one at a time.

This gives you approximately two more rows (two more prediction) for each field. So if you have 10 fields that you think affect the outcome, you will be sending about 20 times more requests. If it is all automated they might not even notice.

Then if, say, one of the fields is age, and you changing it from 21 to 20 makes no difference to the prediction, but 21 to 22 does, then you know there is something sensitive about <=21 vs. >=22 (for that record).

If one of the fields is gender, and you change it from male to female, and get the same prediction, you know it is not important for that record. If it makes no difference to any of your predictions, then it suggests they are not using that field, or it is of very low importance. Which may be a Phew! moment for your manager.

(As you gave no sample data, I used age and gender as simple examples to give you an idea; you obviously need to adapt this to your own domain.)

You can get a lot of data this way, but it will take some effort to draw conclusions from it.

Answer 2

Take a bunch of past forecasts and the corresponding historical data that they were attempting to model. Depending on your need, calculate the RMSE (if you expect that hits and misses will cancel out for your purposes, such as billing cycle budgeting) or the MAE, which is a bit more intuitive to understand.

"A forecast method that minimises the MAE will lead to forecasts of the median, while minimising the RMSE will lead to forecasts of the mean. Consequently, the RMSE is also widely used, despite being more difficult to interpret." https://otexts.com/fpp3/accuracy.html#accuracy

There is also MAPE (mean absolute percentage error). This puts a penalty on negative errors.

If you are doing TSLM (time series linear modeling) by providing some additional data, such as planned production, make sure to take into account differences with actual.

Be realistic about implicit model assumptions, such as the lookback period. A rule of thumb I use is to avoid historical periods longer than about 200 points and use the shortest useful forecast horizon because confidence intervals quickly grow out of any helpful information.

If you have the staff resources and the outside expense savings justifies it, forecasting isn't all that hard to pick up. The text cited above will walk you through the process. I can't recommend trying it on Excel because the time required for QA is inordinate.