# Softmax cost function

In this post, https://cs231n.github.io/linear-classify/#webdemo it mentions "Softmax classifier interprets the scores inside the output vector f as the unnormalized log probabilities." Aren't we taking the log afterward and f is equal to just Wx+b?

No, that is not what is happening, the definition of the Softmax classifier is: $$z \in \mathbb{R}^K; \sigma(z)_j = \frac{e^{z_j}}{\sum_{k = 1}^{K} e^{z_k}}$$ Let's talk first about what the softmax function aims to do, but before we can understand that, we need to understand the aim of classification in general. Classifiers are functions which take a vector in $\mathbb{R^n}$ and output one of $k$ classes. The softmax function takes in a vector $z$ and outputs a value in $\mathbb{R^+}$ whose values sum to one.
Now let's get to your question, if I were to just take $\log(Wx + b)$. It is actually almost exactly that, we just take the exponent so that our values range from $0$ to $1$. However, these values are not normalized and in order to normalize it we need to divide by the sum and we take the exponents to maintain a positive value.
• It is the equivalent of $f$ in the equation you posted, or more precisely, the output of $\log (Wx + b)$ – PSub Jun 18 '17 at 15:20