I have trained a model for one-hot binary prediction for many classes, and am now applying it to the testing set of samples. However, a lot of the predictions for samples are 0 for every class. I'm not sure what to do with these results, as I need to make a confusion matrix (nxn for the number of classes) but I don't know where these predicted-no-class results should go. Do I just discard them? I would imagine that this would create a faulty image of the error rate of the model.
1 Answer
$\begingroup$
$\endgroup$
It depends on the design of your task, there are two options:
- The task is regular multiclass classification, i.e. every instance must belong to exactly one class. In this case it would be a mistake to one-hot-encode the class, it can simply be encoded as an int (for example with LabelEncoder). The model will always predict exactly one class for one instance so the case of zero class is impossible.
- The task is multi-label clasification, i.e. every instance can belong to zero, one or multiple classes. In this case an instance can be predicted as belonging to no class at all, this is normal. In this setting the confusion matrix should not be done with a $n \times n$ matrix across classes, because the classes are independent (btw it's not only about the case of zero class, the case of multiple classes would also be impossible to represent this way). Instead there should be one binary confusion matrix for every independent class.
