Let's consider I trained a model that gives the probability if an apple will rot and deployed it. Once it's deployed, how can I measure/evaluate its performance. At first I thought, we can just check against the actual outcome and see if the prediction was made correctly. However, this model is an preventative model so that person can take action to make sure apple doesn't rot. Since person took some action to prevent the apple from rotting, the accuracy or any other metric will do poorly.

In this case, what would be the best way to evaluate the model as time goes by? How do we check if model is still correctly predicting the outcome of apple?




Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.