I have a neural network model (implemented from scratch) which gives me some continuous outputs and I have used a sigmoid layer, in the end, to convert it into a binary classifier. But my original problem is a multiclass problem. I tried to use the softmax layer but somehow that cannot be applied to my model architecture as the output is not in that format (I get some continuous value as output).
Thus I realized that an alternative way would be to use one versus all classifier but I am not able to figure out how to put that on top of my binary classifier. Please forgive me if my concepts are a bit weak, but I would really be thankful if someone could help me with this.
Like, how can I convert the binary classifier to a multiclass classifier while keeping the original architecture and what kind of loss function should be used? Would that be different from the binary classifier loss function?
Thanks in advance.


1 Answer 1

  • You may keep N neurons and Sigmoid and treat is like multi-label though your data will always be multi-class. So it should learn like multi-class

  • Create your own One-vs-One

    • Have 3 copy of the same model
    • Fit with A|B, B|C and C|A respectively
    • For new data, you need the average of two for each Class probability
  • Use scikit-learn Classifier

    • OneVsOneClassifier Link
    • OneVsRestClassifier Link
  • $\begingroup$ Thanks a lot for your ideas! Regarding the first one, I would be thankful if you could explain it a bit. Like, in this do you suggest I keep one sigmoid? If so, how do I identify the different classes? I read about hierarchical sigmoid, but couldn't understand it as to how to apply it. $\endgroup$
    – Alex
    Commented May 17, 2021 at 18:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.