I have a binary classification problem which in the test set, the number of data in both classes are equal (the test number of class 0 and class 1 are equal). Since we know that the number of samples from every class are equal, I use median on the probabilities of logistic regression output (probabilities of samples for class 1) and map probabilities to zero and one and then calculate f1-micro and f1-macro. But they are absolutely equal and I don't know is it odd or not and why is this happening. I would be grateful if you have any idea what's going on and what is wrong.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
The difference between macro and micro averaging for performance metrics (such as the F1-score) is that macro weighs each class equally whereas micro weights each sample equally. If the distribution of classes is symmetrical (i.e. you have an equal number of samples for each class), then macro and micro will result in the same score.
As an example for your binary classification problem, say we get a F1-score of 0.7 for class 1 and 0.5 for class 2. Using macro averaging, we'd simply average those two scores to get an overall score for your classifier of 0.6, this would be the same no matter how the samples are distributed between the two classes.
If you were using micro averaging, then it would matter what the distribution was. Say that class 1 made up 80% of your data, the formula would then be 0.7*80% + 0.5*(100%-80%) which would equal 0.66, since each sample is weighed equally and as a result the score is representative of the data imbalance. If class 1 made up 50% of your data, the formula would shift to 0.7*50% + 0.5*(100%-50%) which would be 0.6, the same as the result from macro averaging.
If your data was perfectly balanced, then macro and micro averaging will both result in the same score. If not, there's still a chance that they result in the same score depending on the exact distribution of scores (or if your estimator has the same performance for all classes involved).
-
$\begingroup$ I recommend relying on the definition for the metrics in the scikit-learn. Based on these definitions, the definition that you have proposed for micro f-score, is not true. Weighted f-score will encounter the proportion of each class as their weights to calculate the weighted f-score. On the other hand, to calculate the micro f-score, you have to sum up all the TPs, FPs, and FNs as the total TP, FP and FN. Then you use them to calculate the micro f-score. $\endgroup$– elldoraApr 1 at 15:02