0
$\begingroup$

I have dataset that looks like this

F1 F2 F3 F4 F5 F6 A1 A2 A3
1  0  0  0  4  3  X  X  X
0  3  9  0  0  0  X  A  X
0  0  0  0  1  0  X  X  X
0  0  3  2  0  0  X  X  X
0  5  0  0  0  1  X  X  X
0  0  7  0  1  3  X  X  X
0  0  0  1  1  0  X  X  B
1  3  0  2  0  1  X  X  X
7  0  0  2  1  3  X  X  X
4  0  0  0  5  0  B  A  X
0  0  7  2  1  3  X  X  X

where F1 to F6 are the features

and A1 to A3 are the prediction attributes.

as you can see most of the records are X X X

when I build my prediction algorithm and try to evaluate the performance I use f1_score

but due to most records have X X X values, I get high f1_score.

which I think this is not the right way to evaluate.

the main point of the evaluation is to find if the values A B and C are predicted correctly.

$\endgroup$
1
  • $\begingroup$ I replaced the word "accuracy" with "evaluation" in your question, because this word has a specific meaning: accuracy is an evaluation measure which is different from f-score for instance. Feel free to revert the changes if you don't like it. $\endgroup$
    – Erwan
    Mar 29, 2020 at 12:55

1 Answer 1

0
$\begingroup$

It's not possible to obtain a single f1-score when there are several classes, probably you are using micro or macro f1-score?

First don't use the micro F1-score, since it gives the same weight to every instance and therefore gives a lot of weight to the "X" class.

Macro F1-score should be closer to what you want to evaluate, since it gives every class equal weight. If you want you could even calculate the macro F1-score over only classes A,B,C (i.e. excluding X), but that seems unnecessary to me.

$\endgroup$
2
  • $\begingroup$ how can I do f1-score for ABC only? $\endgroup$
    – asmgx
    Mar 29, 2020 at 17:41
  • $\begingroup$ f1 score is for a single class, for instance you calculate the f1-score for individual class A, B, C or X, it's a different score for each of them. Normally the macro f1-score is the mean of all the individual f1-scores across all classes, but instead you could do it across classes A,B,C only. $\endgroup$
    – Erwan
    Mar 29, 2020 at 17:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.