I came across this paper by some Facebook researchers where they found that using a softmax and CE loss function during training led to improved results over sigmoid + BCE. They do this by changing the one-hot label vector such that each '1' is divided by the number of labels for the given image (e.g. from [0, 1, 1, 0] to [0, 0.5, 0.5, 0]).

However, they do not mention how this could then be used in the inference stage, because the required threshold for selecting the correct labels is not clear.

Does anyone know how this would work?

  • $\begingroup$ As mentioned in the paper, the value of k i.e the number of hashtags per image, is required to create the target vector against which the CE loss is calculated for every batch. Note, the model is still predicting a probability distribution for the labels ( hashtags ). So, during inference, you can select the top N labels which have the highest probability. $\endgroup$ – Shubham Panchal Apr 8 at 7:47
  • $\begingroup$ Yes but how do I decide what to set as N when selecting the top N labels in inference? My images can have 1, 2, or maybe 3 labels per image during inference, so typically I used sigmoid and set a threshold to 0.5, but with Softmax I cannot do this since the sum of all probabilities will equal 1. If I do as you say and choose the top N, what criteria would I use to determine the number of labels to choose? Because if one image should only have 1, whereas the next image should have 3 labels, there is no way that I can think of that would allow me to dynamically choose the value of N per image. $\endgroup$ – Steve Ahlswede Apr 8 at 8:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.