Timeline for What does it mean that classes are mutually exlcusive but soft-labels are accepeted?
Current License: CC BY-SA 4.0
14 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 23, 2018 at 13:09 | history | edited | Green Falcon | CC BY-SA 4.0 |
edited body
|
| May 23, 2018 at 13:03 | comment | added | ignatius | Let us continue this discussion in chat. | |
| May 23, 2018 at 13:03 | comment | added | ignatius | Well, then It has to be taken that mutuallu-exclusive classes and valid probability distribution for labels, yields to one-hot encoding, though it is not specified nor documented. Furthermore, the API does not raise any error if the labels are not one-hot encoded. I think it is very ambiguous | |
| May 23, 2018 at 13:01 | comment | added | Green Falcon | About To be precise, an example: for one instance, one m... I didn't understand what you meant. The prediction is wrong but what network does is true. You train more for better results. | |
| May 23, 2018 at 12:59 | comment | added | Green Falcon | mutually exclusive part implies one-hot-encoding. Moreover, it has been referred to in the link I have provided here. | |
| May 23, 2018 at 12:58 | comment | added | ignatius | To be precise, an example: for one instance, one may have labels =[1 0 0], one-hot encoding for this instance, saying that it belongs to class 0. logits =[1 2 3], ouput class-scores (or unscaled log probablities) of the last layer of the NN. Then, for classification, softmax(logits) = [0.0900 0.2447 0.6652]. In this case, the predicted class is 2, miss-classification. But, technically, the documentation allows labels = [0.5 0.3 0.2] , some king of likelihood, that does not make much sense in a multi-class case | |
| May 23, 2018 at 12:56 | comment | added | ignatius | That's what exactly I am askign.... because the documentation does not say that labels have to be one-hot encoded, but a valid probability distribution... finally I think you understand my question, and you come up with the same doubt... | |
| May 23, 2018 at 12:54 | comment | added | Green Falcon | I didn't understand! can you give an example of a valid probability distribution for the labels that are not one-hot encoded? for multi label class classification, the last layer should have sigmoid non-linearity and each output may have different value between 0 to 1 which represents the probability of the existance of the corresponding component in the input. The last layer's output should not be summed in those cases. | |
| May 23, 2018 at 12:48 | comment | added | ignatius | I know that, but this is not what we are discussing here... the input tensor labels for softmax_cross_entropy_with_logits are the ground truth annotation of each training instance... Yes, the softmax turns the logits to a valid distribution, but again, logits are logits and labels are labels (ground-truth annotations) | |
| May 23, 2018 at 12:44 | comment | added | Green Falcon | It is a valid distribution because you have passed the logits, the linear part of the last layer for each neuron, as the input to the softmax layer. softmax layer is a mapping. $softmax:R^m−>R^m$. What softmax does is normalizing the inputs to find the probability for each output. Consequently, whatever you have in the last layer's logits, the softmax layers turns them to valid distribution, the sum of all outputs will be equal to one. This is also true for mutually exclusive labels. They are one-hot-encoded and they sum to one. | |
| May 23, 2018 at 12:37 | comment | added | ignatius | I'm not completely agree, sorry... Although in While the classes are mutually exclusive, their probabilities need not be referees to the probabilities of the predictions, in All that is required is that each row of labels is a valid probability distribution does not say that the labels have to be one-hot encoded, but a valid probability distribution. It is confusing for me, can you give an example of a valid probability distribution for the labels that are not one-hot encoded? | |
| May 23, 2018 at 12:31 | comment | added | Green Falcon | @ignatius While the classes are mutually exclusive, their probabilities need not be. This sentence is for prediction time. Moreover, the labels you provide to your network should be strictly defined. It means they should have a specified label. If you provide probabilities with ambiguity, you can not expect your network to find useful things. | |
| May 23, 2018 at 12:23 | comment | added | ignatius | I appreciate your response, but I think your are answering it wrong, at least for the first two paragraphs. I feel you're confusing the labels with the predictions. I understand that for a multi-class classifier, the predictions are probabilities, given by the softmax, for example, But it is not he point, the question is how can soft-labels be accepted in a multi-class exclusive case. | |
| May 23, 2018 at 12:19 | history | answered | Green Falcon | CC BY-SA 4.0 |