Can someone explain me, how to classify a data like MNIST with MLBP-Neural network if I make more than one output (e.g 8), I mean if I just use one output I can easily classify the data, but if I use more than one, which output should I choose ?

  • $\begingroup$ en.wikipedia.org/wiki/Winner-take-all $\endgroup$
    – Yavar
    Jun 10, 2014 at 8:45
  • $\begingroup$ Please, add information/reference link on that MNIST data, so as to make your post self-contained. Thanks. $\endgroup$
    – Rubens
    Jun 20, 2014 at 6:19

2 Answers 2


Suppose that you need to classify something in K classes, where K > 2. In this case the most often setup I use is one hot encoding. You will have K output columns, and in the training set you will set all values to 0, except the one which has the category index, which could have value 1. Thus, for each training data set instance you will have all outputs with values 0 or 1, all outputs sum to 1 for each instance.

This looks like a probability, which reminds me of a technique used often to connect some outputs which are modeled as probability. This is called softmax function, more details on Wikipedia. This will allow you to put some constraints on the output values (it is basically a logistic function generalization) so that the output values will be modeled as probabilities.

Finally, with or without softmax you can use the output as a discriminant function to select the proper category.

Another final thought would be to avoid to encode you variables in a connected way. For example you can have the binary representation of the category index. This would induce to the learner an artificial connection between some outputs which are arbitrary. The one hot encoding has the advantage that is neutral to how labels are indexed.


The algorithm that is used in this case is called one-vs-all classifier or multiclass classifier.

In your case you have to take one class, e. g. number 1 , mark it as positive and combine the rest seven classes in one negative class. The neural network will output the probability of this case being class number 1 vs the rest of the classes.

Afterwords, you have to assign as positive another class, e.g. number 2, assign all other classes as one big negative class and get the predicted probability from the network again.

After repeating this procedure for all eight classes, assign each case to the the class that had the maximum probability from all the classes outputted from the neural network.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.