If you have a text in positive label, and your model think it is positive then the positive probability your model output will be the largest.
If you ask your model which is the second most likely label that you(your model) think this text sample belong to, your model's answer is the class that has the second largest probability in the output, and so on.
In summary, your model rank the class from most likely to less likely to your sample. So the order of the probabilities depend on your model belief, such that the most likely class will has the largest probability and the least likely class will has the least probability.
My question: suppose that we have a piece of text in Positive label. So, do we have to have these probabilities in this order:
P(pos) > P(neu) > P(neg)
Not exactly, it depends on your model belief, which depends on how good is your data to express the idea of positive, neutral and negative. But usually when use logistic regression to classify 3 class positive, neutral and negative, people will set a threshold for positive, neutral and negative in the probability range, for example: > 0.7 is positive, in [0.4, 0.7] is neutral and the remaining is negative. By doing this, we implicitly assume that the probabilities are indeed have order as you said. This is because we assume that there is an order between positive, neutral and negative, such that neutral is between positive and negative. But if we are dealing with another problem for example classify dog, cat and fist, then I don't think we can assume the order.
What does it mean when we have them in this order:
P(pos) > P(neg) > P(neu)
It means that the model believe the most likely class to your sample is positive, the second most likely is negative, and the least likely is neutral.
Can we conclude anything from this? For example, can we say with confidence that the label is Positive like as before?
In my opinion, the model is confident with its answer, if we choose to believe it, then we can confidently say that the sample's class is positive as before.
