2
$\begingroup$

I am training a classifier (e.g, animals), and since many categories are too similar (e.g., insects), I am grouping some categories together (e.g., bugs and mosquitoes as insects). Then, I will train a classifier to distinguish between the different insects.

How can I evaluate the performance of the two together?

Let's say that the first level is ['cat', 'dog', 'insects'] with accuracy of 90%

And the second level is ['mosquito', 'bug'] with accuracy of 80%

What's the overall accuracy?

$\endgroup$

1 Answer 1

0
$\begingroup$

First mind that accuracy is not a very good performance measure in case there is some imbalance between the classes. Micro or macro F1-score are more informative.

Any overall performance measure can be obtained by taking the full set of instances, considering only the final predicted label against the true label for every instance. From there a confusion matrix can be obtained the usual way. The intermediate level (e.g. "insect") does not matter. In the case of accuracy you just need to count for every instance whether the true label is the same as the final predicted label.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.