I am little bit confused on why the scoring function that is the $f(X,W)$ is chosen to be $W,X$ while they talk about Softmax and SVM loss in this.

Can't they take Softmax classifier or SVM classifier and then explain the losses?

Was there a particular need of taking the above mentioned scoring function?

  • $\begingroup$ I don't see where they actually define a linear scoring function. They talk about hinge loss and cross-entropy loss, which are calculated as a function of X and W, but are not a simple linear combination of X and W. $\endgroup$ – StatsSorceress Jan 9 '18 at 17:35

The notation they're using is a bit funny. $f$ is just the dot product between the input and weights, $w^Tx_i$. The loss function is what differentiates those classifiers. Each loss function takes that dot product as an input and uses it in different ways. All of those loss functions compare the output of $f$ in some way with an associated $y$ value: you are computing your loss relative to a target variable to understand how well your model is performing, and then you can compute the gradient of the loss function to improve those predictions. If you ignore the loss function, you're just left with linear regression predictions (and even linear regression needs a loss function, MSE, to fit the weights) which aren't being evaluated relative to anything.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.