error useing soft max gives outputs greater than 1

Question

I am using Hugging Face AutoModelForSequenceClassification, model is roberta, using it for text classification.

There are 3 classes.

The output is: [-3.7550,-4.4172,7.8079]

I need to convert this to probabilities should I apply soft max to this to get the probabilities , if i do that i am getting outputs greater than one

[9.51,4.90,0.99]

You can indeed use the softmax functions to get "probabilities", however I think you are missing the exponents since the none of the individual values are greater than one (they should be 9.51e-6 en 4.90e-6 respectively). — Oxbowerce, Jan 11 at 12:47

noe · Accepted Answer · 2023-01-11 15:02:54Z

This piece of Python code is what you described:

import torch
a = torch.tensor([[-3.7550,-4.4172,7.8079]])
b = torch.softmax(a, 1)
print(b)
print(torch.sum(b))

It will print the following:

tensor([[9.5124e-06, 4.9057e-06, 9.9999e-01]])
tensor(1.)

From this, we know two things:

Softmax works as intended, and the resulting values add up to 1.
The result of the softmax is $9.5124 \cdot 10^{-6}$, $4.9057 \cdot 10^{-6}$ and $9.9999 \cdot 10^{-1}$. You, however, did not take into account the exponents of those numbers, and that is why they don't add up to 1 in your computation.

1 Answer 1