2
$\begingroup$

I still novice in machine learning. My problem is as follow :

I use two binary logistic regression models for making comparaison. Additionally to LogLoss and accuracy, I would like to use for metric : precision / recall/ F1-mesure.

I have used the formulat :

print (classification_report(y_test, y_pred.round()))

My question is: which one, for the tree metric value, should I used. is it the weighted average ore the parametres of the true values (number 1 in picture)

enter image description here

$\endgroup$
1
$\begingroup$

The performance measure should be related to the actual real world problem you face. (Imagine, you are a football trainer. You are seeking for young talents to introduce them to your very expensive training camp. It might be more important to you to prevent expenses on kids, that proof not to be as talented as hoped for, than reaching all hidden talents with the program. So you might be interested in strong precision of beforehand assessement. But if you dont have to care about funding, your priorities might change. You could be more interested in finding any last talented kid. Then you would be more interested in recall.)

If you want good probabilty predictions, use logloss. If you want the prediction score to separate very well among classes, rather use area under RO curve. Your choice should depend on your real world cost for missclassification or bad predictions.

The averages are merely an aggregation of the class specific statistics. This is mostly useful for multi class problems. For two class problems I recommend to look at the class specific statistics instead. Anywasy, how averageing is conducted, you can read here: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html

$\endgroup$
1
  • $\begingroup$ thank you very much for your answer. it is so helpful $\endgroup$ Jan 29 '20 at 16:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.