I am looking for tools that allow me to monitor machine learning models once they are gone to production. I would like to monitor:
- Long term changes: changes of distribution in the features with respect to training time, that would suggest retraining the model.
- Short term changes: bugs in the features (radical changes of distribution).
- Changes in the performance of the model with respect to a given metric.
I have been looking over the Internet, but I don't see any in-depth analysis of any of the cases. Can you provide me with techniques, books, references or Software?