0
$\begingroup$

Why is it more beneficial to automatically train a machine model by a conversion rate or by a KPI instead of by periodic time?

The following image shows an example of the conversion rate. However, as soon as the conversion rate drops, the model is automatically re-trained. As soon as the conversion rate drops to a defined minimum, re-training is started. Why is this approach more beneficial than saying every week on Sunday?

enter image description here

$\endgroup$

1 Answer 1

2
$\begingroup$

I will answer in 2 parts-

Why retrain based on KPI

In real world, once you deploy an ML model in production, you would want it to perform to the best of its capability so you (or your client) can make better decisions related to your (client's) business. As long as the model's KPI is good, it is performing well. Suppose it drops below a certain threshold and you don't retrain, then you will not be able to make better decisions based on the models' results which could potentially impact your business. So a KPI is preferred for model monitoring.

Why not train periodically

In today's world, data can change pretty quickly. If you choose a period which is too large compared to the rate of the data change, your model will not be able to capture the current trends properly; and again, it could impact your business. If you choose a period which is too small compared to the rate of the data change, retraining model frequently could be expensive (which would incur more cost); and again, it could impact your business. So it is better to not train the model periodically.

All in all, a KPI is generally preferred as you can avoid unnecessarily retraining the model and also you can visualize your model's performance based on the indicator/metric.

$\endgroup$
3
  • 1
    $\begingroup$ Thank you very much, that helps me a lot! The answer is very detailed and perfect. Is there also a scientific reference for this ? $\endgroup$
    – Test
    Oct 18, 2022 at 18:04
  • $\begingroup$ I do not have any reference as of now. But you can search if you want. There are many Data Science websites available which could give information on model monitoring. $\endgroup$
    – Ankit Seth
    Oct 19, 2022 at 15:24
  • $\begingroup$ I have done a lot of searching on this and am looking for a scientific paper on this that confirms your statement. Until yet, I have not been able to find a source for it. $\endgroup$
    – Test
    Oct 19, 2022 at 17:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.