Question: A image has multiple labels. Given a set of image with labels, how to adapt the softmax layers?

My idea:

  • Encode multiple labels to 0-1 variables. Use logistics regression as the output layer.
  • Choose the top X labels in softmax output. But I do not know
    1. how to determine the threshold.
    2. how to modify the code in a pretrained net. Let's say
      "op": "null",
      "param": {},
      "name": "softmax_label",
      "inputs": [],
      "backward_source_id": -1
      "op": "Softmax",
      "param": {"grad_scale": "1"},
      "name": "softmax",
      "inputs": [[510, 0], [511, 0]],
      "backward_source_id": -1

Any recommendations on tutorials are appreciated. I have some theory background from the book ESL.


3 Answers 3


I don't know what's going on in your code, but you seem to be close: to get multiple labels, simply replace the softmax output layer with a logistic layer (or something else that maps a real number to a probability), then optimize the cross entropy. That way you will have a probability associated with each label such that their sum across labels no longer need to add to unity.

  • $\begingroup$ You helped me get more clear about the idea. Any recommendation on real tutorials, examples or codes? $\endgroup$ Feb 26, 2016 at 0:58
  • $\begingroup$ Maybe you should accept this answer instead, then? $\endgroup$
    – kbrose
    Feb 22, 2018 at 0:21

Replace softmax activation with sigmoid activation function in the last layer. Sigmoid convert the scores to the range of [-1 to 1]. Then you can apply the thresholds on this score value.

  1. While training, you have to consider the multiple ground truths for that image.

  2. Then while evaluating, you will get multiple predictions of the image unlike softmax which makes sure the sum of the probabilities of each class is equal to 1.


I have read related tutorials over the past several days.

There are two possible solutions

  1. transfer the problem into several multi-classication problem.
  2. use algorithms that can handle multi-label problem.

On the software side:

  1. extract network parameters except the last layer.
  2. use a SVM(or other methods) to handle the last layer.


  1. stanford computer vision online course
  2. caffe tutorial

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.