Time of trainig vs time of prediction, which one is used during classification algorithms comparison?

Question

I need to use many algorithms for making a binary classification, such as Logistic regression, SVM, XGBoost, CatBoost, ... I get an interesting improvement but All of those algorithms (except LR) take a very long time for training data.

So I need to know if I must indicate training time in my comparison or it is doesn't need to be indicated and predicting time is enough ??

And how many time tuning parameters and fitting training should be done in a real application? does it done each time we insert a new row on data?

Answer 1

Since part 1 of your questions has been answered, here is the answer to part 2-

How often should the model be re-trained (or refreshed) and how often should the hyperparamters be tuned? It depends on the problem that you are trying to handle with the model. Example- if a model is used to predict fraud bookings then I would like to re-train it every month (at the maximum) because I think that is how often one can expect new fraud trends to arise. So depending on how often you expect your data to change, re-train accordingly. And whenever you re-train, you tune the hyperparameters as well.

Answer 2

I get this answer, if it may helping you or if you have any other information to add :

If we are talking only about computational time then prediction time is more important if the training time is finite and accuracy is comparable. Training time can be handled easily by using a good machine.

This is important because any machine learning system is expected to generalize or at least have higher accuracy on the testing set. To achieve this better performance training time is often not considered. If one algorithm has significantly higher training time but good accuracy on testing test then it is considered better because once the training is done, the algorithm doesn’t need much tweaking. Obviously in this case we assume that the training time is finite which can be reduced by using a better machine.