-2
$\begingroup$

When computing loss functions, people use $(target-actual)^2$. They sqaure it to prevent any negative loss. But we can even use $|(target-actual)|$ to prevent any negative loss. So, why do people prefer the first option more than the second?

$\endgroup$
1
$\begingroup$

Apart from the correct answers which you find in the comment section, you mention "the square [..] prevent any negative loss".

In principle you can also have a negative loss. The point is without the square, you have $(x-y) \neq (y-x)$ for $x \neq y$. In particular, the loss would not be symmetric and for $x = 0$, you have $(x-y) = -y$. So by increasing $y$ you decrease the loss. The loss would thus not be lower bounded so that there is no global minimum for the loss.

| improve this answer | |
$\endgroup$
  • $\begingroup$ You said 'In principle you can also have a negative loss', I don't think we can have negative losses because then the negative losses could cancel out with the positive losses. $\endgroup$ – Dhruv Agarwal Sep 23 at 13:22
  • $\begingroup$ If you minimize $\sum_{i} ||\mathrm{target}_{i}-\mathrm{actual}_{i}||_{2}^{2}-\theta_{i}$, where $\theta = \theta_{i} > 0$ is fixed. you will get the same optimal result and the loss can have negative values. $\endgroup$ – Graph4Me Consultant Sep 23 at 13:24
  • $\begingroup$ ok, may be replace "loss" with "objective function". But still my point is that without the square, computing $x-y$ is completely wrong (and this is not due to negative values) but as it is not measuring anything usefull. $\endgroup$ – Graph4Me Consultant Sep 23 at 13:41
  • $\begingroup$ For example, you could have a neural network that minimizes or maximizes the cosine similarity. In this case, the optimal objective value is either 1 or -1. $\endgroup$ – Graph4Me Consultant Sep 23 at 13:44

Not the answer you're looking for? Browse other questions tagged or ask your own question.