2
$\begingroup$

When dealing with classification for multiple classes present in the same sample, can the output layer have the form of one-hot encoding, but instead of only one hot, have multiple? That is, in case of a only single class being present in the sample, the encoding could be [0,1,0]. For multiple classes in the same sample, can it be [0,1,1]? And if yes, what loss function should be used?

$\endgroup$
2
  • 1
    $\begingroup$ I am a bit confused. When you say input, du you mean sample? To my understanding, you are asking about samples that can belong to multiple multiple classes? Am I correct? $\endgroup$
    – Broele
    Mar 10, 2023 at 20:23
  • $\begingroup$ Yes, by input I mean sample. And also yes, the same sample belongs to more than one class. $\endgroup$
    – smone
    Mar 10, 2023 at 20:33

1 Answer 1

5
$\begingroup$

Yes, it is possible to do it exactly as you describe it. This is called multilabel-classification.

If doing so, you would treat each element of the output as an independent prediction of a binary classification problem, i.e. the first element would predict, if the sample belongs to class 1, the second element would predict if the sample belongs to class 2 and so on.

As a loss function, you could take the sum of the cross-entropy loss per output element. If you have $m$ classes, $y=(y_1,\ldots, y_m)$ would be the binary target and $\hat{y}=(\hat{y}_1,\ldots,\hat{y}_m)$ would be your prediction, then the loss could be $$\mathcal{L}=\sum_{j=1}^m -y_j\log \hat{y}_j - (1-y_j)\log (1-\hat{y}_j)$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.