I have a multi-class classification problem with class imbalance. I searched for the best metric to evaluate my model. Scikit-learn has multiple ways of calculating the F1 score. I would like to understand the differences.

What do you recommending when there is a class imbalance?


1 Answer 1


F1Score is a metric to evaluate predictors performance using the formula

F1 = 2 * (precision * recall) / (precision + recall)


recall = TP/(TP+FN) and precision = TP/(TP+FP)

and remember:

enter image description here

When you have a multiclass setting, the average parameter in the f1_score function needs to be one of these:

  • 'weighted'
  • 'micro'
  • 'macro'

The first one, 'weighted' calculates de F1 score for each class independently but when it adds them together uses a weight that depends on the number of true labels of each class:


therefore favouring the majority class.

'micro' uses the global number of TP, FN, FP and calculates the F1 directly:


no favouring any class in particular.

Finally, 'macro' calculates the F1 separated by class but not using weights for the aggregation:


which resuls in a bigger penalisation when your model does not perform well with the minority classes.

The one to use depends on what you want to achieve. If you are worried with class imbalance I would suggest using 'macro'. However, it might be also worthwile implementing some of the techniques available to taclke imbalance problems such as downsampling the majority class, upsampling the minority, SMOTE, etc.

Hope this helps!

  • 1
    $\begingroup$ I do already downsampling on the training set, should I do it also on the testset? I'm really confuse on witch dataset should I do all the technique for taclke imbalance dataset. $\endgroup$
    – Fractale
    Commented Nov 9, 2018 at 2:35
  • 1
    $\begingroup$ You want to avoid downsampling on the test set because it will artificially bias your metrics for evaluating your model's fit, which is the point of the test set. $\endgroup$
    – kevins_1
    Commented Nov 9, 2018 at 19:03
  • $\begingroup$ this is a great answer... $\endgroup$ Commented Nov 11, 2020 at 12:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.