Loss functions are important part of machine learning.

Square loss is one of the most popular loss functions.

Mean squared error (MSE) measures the average squared difference between the estimated values and what is estimated

The estimated value is denoted as $\hat y$, what is estimated is denoted as $y$.

$${\displaystyle \operatorname {MSE} ={\frac {1}{m}}\sum _{i=1}^{m}(y_{i}-{\hat y_{i}})^{2}.}$$

in the context of distance/difference, it seems that MSE and square loss do the same job.

what is the relationship between "square loss" and "Mean squared error"?


$$V(f(x), y) = (1-yf(x))^2$$

In the context of classification where $y_i$ is $1$ or $-1$:

\begin{align} MSE &= \frac1m \sum_{i=1}^m (y_i-f(x_i))^2 \\ &= \frac1m \sum_{i=1}^m y_i^2 (1-y_if(x_i))^2\\ &= \frac1m \sum_{i=1}^m (1-y_if(x_i))^2\\ &= \frac1m \sum_{i=1}^m V(f(x_i), y_i) \end{align}

  • $\begingroup$ Generally we use MSE for regression problems. Like to know if both of the equations are still same? Earlier I thought square loss will return list of values, whereas MSE is just a mean of these square loss value. $\endgroup$ – vipin bansal Jun 4 '19 at 8:05
  • $\begingroup$ I think that depends on the definition of square loss, the form of. $(1-yf(x))^2$ suggest classificaiton. Also, the wiki page is indeed just discussing loss function for classification. If we define square loss for regressionto be $(y_i-f(x_i))^2$, then the relation still hold. $\endgroup$ – Siong Thye Goh Jun 4 '19 at 8:11
  • $\begingroup$ thanks for your answer. would you please give more detail from first line to second line? $$ \begin{align} \operatorname {MSE} &={\frac {1}{m}}\sum _{i=1}^{m}(y_{i}-{\hat y_{i}})^{2} \\ &= \frac1m \sum_{i=1}^m y_i^2 (1-y_if(x_i))^2 \end{align} $$ $\endgroup$ – JJJohn Jun 4 '19 at 8:14
  • $\begingroup$ I just factorize $y_i$ out of the square and note that $\frac1{y_i} = y_i$. $\endgroup$ – Siong Thye Goh Jun 4 '19 at 8:15

Square loss is the overall amount of error in your model. However, it depends on the number of observations you have. A loss of a billion might not be so bad if you have a trillion observations. On the other hand, a loss of 500 might be awful if you have 25 observations. MSE accounts for the number of observations by averaging the loss over all of the observations. The MSE in the former example is 0.001, while the MSE in the latter example is 20.

If you think about a classification problem, square loss would be analogous to the number of misses, while MSE would be analogous to the accuracy. You will have more incorrect classifications if you attempt more classifications, but this doesn't mean that your model performance is any worse when you attempt many classifications.

Except for numerical technicalities on a computer, you should find the same model by minimizing the square loss or MSE loss functions, so in some sense, you're right to say that they "do the same job."


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.