My desired output is not 1-hot encoding, but
like a 10 D vector: [1, 0, 1, 0, 1, 0, 0, 1, 1, 1]
and the input is like the normal MNIST data set.
I want to use TensorFlow to build a model to learn this, then which loss function should I choose?
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ It looks like you are performing multilabel classfication; e.g., if there are potentially multiple digits in one output. If so, use the cross entropy. $\endgroup$– EmreCommented Nov 16, 2016 at 5:22
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
If your classes arre not mutually exlcusive, then you just have multiple sigmoid outputs (instead of softmax function as seen in example MNIST classifiers). Each output will be a separate probability that the network assigns to membership in that class.
For a matching loss function, in TensorFlow, you could use the built-in tf.nn.sigmoid_cross_entropy_with_logits - note that it works on the logits - the inputs to the sigmoid function - for efficiency. The link explains the maths involved.
You will still want a sigmoid function on the output layer too, for when you read off the predictions, but you apply the loss function above to the input of the sigmoid function. Note this is not a requirement of your problem, you can easily write a loss function that works from the sigmoid outputs, just the TensorFlow built-in has been written differently to get a small speed boost.
answered Nov 16, 2016 at 7:58
-
$\begingroup$ I use tf.nn.sigmoid_cross_entropy with_logits ,but my output looks quite bad , nearly all elements in the output is between 0.2-0.4,no higher than 0.5 . the structure of my neural network: output is a set of 64*64*3 channels image , the labels is a set of 4096(64*64) vectors (each element is 0 or 1),the desire output is a set of 4096 vectors..I do 1 convolution layer->max pooling->2 convolution layer ,the activation function is all relu ,and finally a fully connected layer (sigmoid) ...and the output is like I say above....Do you have any ideal or suggestion where I'm wrong? thanks $\endgroup$– nsknslCommented Nov 17, 2016 at 11:09
-
$\begingroup$ @特馒头的馒头 That seems like a different question. I cannot debug your ML problem in comments. If this answer helped at all, then please consider accepting it. If it didn't help, then I am sorry but I cannot go into detail of your whole project. Either way, you will be better off describing your new problem in more detail in a new question. $\endgroup$ Commented Nov 17, 2016 at 12:17