5
$\begingroup$

Loss functions are important part of machine learning.

Square loss is one of the most popular loss functions.

Mean squared error (MSE) measures the average squared difference between the estimated values and what is estimated

The estimated value is denoted as $\hat y$, what is estimated is denoted as $y$.

$${\displaystyle \operatorname {MSE} ={\frac {1}{m}}\sum _{i=1}^{m}(y_{i}-{\hat y_{i}})^{2}.}$$

in the context of distance/difference, it seems that MSE and square loss do the same job.

what is the relationship between "square loss" and "Mean squared error"?

$\endgroup$

3 Answers 3

2
$\begingroup$

Square loss is the overall amount of error in your model. However, it depends on the number of observations you have. A loss of a billion might not be so bad if you have a trillion observations. On the other hand, a loss of 500 might be awful if you have 25 observations. MSE accounts for the number of observations by averaging the loss over all of the observations. The MSE in the former example is 0.001, while the MSE in the latter example is 20.

If you think about a classification problem, square loss would be analogous to the number of misses, while MSE would be analogous to the accuracy. You will have more incorrect classifications if you attempt more classifications, but this doesn't mean that your model performance is any worse when you attempt many classifications.

Except for numerical technicalities on a computer, you should find the same model by minimizing the square loss or MSE loss functions, so in some sense, you're right to say that they "do the same job."

$\endgroup$
1
$\begingroup$

$$V(f(x), y) = (1-yf(x))^2$$

In the context of classification where $y_i$ is $1$ or $-1$:

\begin{align} MSE &= \frac1m \sum_{i=1}^m (y_i-f(x_i))^2 \\ &= \frac1m \sum_{i=1}^m y_i^2 (1-y_if(x_i))^2\\ &= \frac1m \sum_{i=1}^m (1-y_if(x_i))^2\\ &= \frac1m \sum_{i=1}^m V(f(x_i), y_i) \end{align}

$\endgroup$
4
  • $\begingroup$ Generally we use MSE for regression problems. Like to know if both of the equations are still same? Earlier I thought square loss will return list of values, whereas MSE is just a mean of these square loss value. $\endgroup$ Jun 4, 2019 at 8:05
  • $\begingroup$ I think that depends on the definition of square loss, the form of. $(1-yf(x))^2$ suggest classificaiton. Also, the wiki page is indeed just discussing loss function for classification. If we define square loss for regressionto be $(y_i-f(x_i))^2$, then the relation still hold. $\endgroup$ Jun 4, 2019 at 8:11
  • $\begingroup$ thanks for your answer. would you please give more detail from first line to second line? $$ \begin{align} \operatorname {MSE} &={\frac {1}{m}}\sum _{i=1}^{m}(y_{i}-{\hat y_{i}})^{2} \\ &= \frac1m \sum_{i=1}^m y_i^2 (1-y_if(x_i))^2 \end{align} $$ $\endgroup$
    – JJJohn
    Jun 4, 2019 at 8:14
  • $\begingroup$ I just factorize $y_i$ out of the square and note that $\frac1{y_i} = y_i$. $\endgroup$ Jun 4, 2019 at 8:15
0
$\begingroup$

Will, there be a difference MSE and SEif we take the square error of whole data? Secondly, in vectorized implementation of a neural network, where we will divide the batch size to take MSE.

Considering Mini-Batch Gradient Descent? global void loss(double* X, double* Y, double *Z, size_t n) {

size_t index = blockIdx.x * blockDim.x + threadIdx.x;

if (index < n) {
    Z[index] = ((X[index] - Y[index]));
}

}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.