0
$\begingroup$

Hinge loss is usually defined as $$L(y,\hat{y}) = max(0,1-y\hat{y}) $$

What I don't understand is why are we comparing zero with $1-y\hat{y}$ instead of some other constant. Why not make it $2-y\hat{y}$, or $\sqrt2-y\hat{y}$ or just take $y\hat{y}$, to check if the observation would be on the right side of the hyperplane? Is there any reason behind '1' as a constant?

Thanks

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

There's no particular reason. It needs a constant different from zero, and 1 fits nicely due to the fact that anything multiplied by 1 is the same thing. You'd get the same result if you replace it with a different number everywhere and adjust the regularization.

Improve this answer
$\endgroup$
3
  • $\begingroup$ But how the behavior of the loss would change if constant is zero? $\endgroup$
    – dmonkoff
    Sep 8, 2019 at 18:55
  • $\begingroup$ If you remove that positive constant, the optimal result is for all the coefficients and predictions to be zero. $\endgroup$
    – anymous.asker
    Sep 9, 2019 at 6:11
  • $\begingroup$ Okay, that makes sense, thanks $\endgroup$
    – dmonkoff
    Sep 9, 2019 at 13:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.