Skip to main content
Data Science

Return to Answer

added 3 characters in body
Source Link
Dave
Dave
  • 4.2k
  • 1
  • 9
  • 32

Square loss (MSE or SSE) does this. Let $y_i$ be an actual value and $\hat{y}_i$ be its estimated value (prediction).

$$SSE = \sum (y_i -\hat{y}_i)^2$$

$$MSE=\dfrac{SSE}{n}$$

Except for numerical issues of doing math on a computer, these are optimized at the same parameter values of your neural network.

The squaring is critical. If a prediction is off by 1 unit, it incurs one unit of loss. If the prediction is off by 2, instead of incurring 2 units of loss, there are 4 units of loss—much worse than being off by 1. If the prediction is off by 3, wow—9 units of loss!

(If you look at statistics literature or some Cross Validated posts, you may see $n-p$ in the denominator of MSE, where $p$ is the number of parameters in the regression equation. This does not change the optimal value but does have some advantages in linear regression, chiefly that it is an unbiased estimate of the error variance under common assumptions for linear regression that you are unlikely to make in a neural network problem.)

Square loss (MSE or SSE) does this. Let $y_i$ be an actual value and $\hat{y}_i$ its estimated value (prediction).

$$SSE = \sum (y_i -\hat{y}_i)^2$$

$$MSE=\dfrac{SSE}{n}$$

Except for numerical issues of doing math on a computer, these are optimized at the same parameter values of your neural network.

The squaring is critical. If a prediction is off by 1 unit, it incurs one unit of loss. If the prediction is off by 2, instead of incurring 2 units of loss, there are 4 units of loss—much worse than being off by 1. If the prediction is off by 3, wow—9 units of loss!

(If you look at statistics literature or some Cross Validated posts, you may see $n-p$ in the denominator of MSE, where $p$ is the number of parameters in the regression equation. This does not change the optimal value but does have some advantages in linear regression, chiefly that it is an unbiased estimate of the error variance under common assumptions for linear regression that you are unlikely to make in a neural network problem.)

Square loss (MSE or SSE) does this. Let $y_i$ be an actual value and $\hat{y}_i$ be its estimated value (prediction).

$$SSE = \sum (y_i -\hat{y}_i)^2$$

$$MSE=\dfrac{SSE}{n}$$

Except for numerical issues of doing math on a computer, these are optimized at the same parameter values of your neural network.

The squaring is critical. If a prediction is off by 1 unit, it incurs one unit of loss. If the prediction is off by 2, instead of incurring 2 units of loss, there are 4 units of loss—much worse than being off by 1. If the prediction is off by 3, wow—9 units of loss!

(If you look at statistics literature or some Cross Validated posts, you may see $n-p$ in the denominator of MSE, where $p$ is the number of parameters in the regression equation. This does not change the optimal value but does have some advantages in linear regression, chiefly that it is an unbiased estimate of the error variance under common assumptions for linear regression that you are unlikely to make in a neural network problem.)

added 443 characters in body
Source Link
Dave
Dave
  • 4.2k
  • 1
  • 9
  • 32

Square loss (MSE or SSE) does this. Let $y_i$ be an actual value and $\hat{y}_i$ its estimated value (prediction).

$$SSE = \sum (y_i -\hat{y}_i)^2$$

$$MSE=\dfrac{SSE}{n}$$

Except for numerical issues of doing math on a computer, these are optimized at the same parameter values of your neural network.

The squaring is critical. If a prediction is off by 1 unit, it incurs one unit of loss. If the prediction is off by 2, instead of incurring 2 units of loss, there are 4 units of loss—much worse than being off by 1. If the prediction is off by 3, wow—9 units of loss!

(If you look at statistics literature or some Cross Validated posts, you may see $n-p$ in the denominator of MSE, where $p$ is the number of parameters in the regression equation. This does not change the optimal value but does have some advantages in linear regression, chiefly that it is an unbiased estimate of the error variance under common assumptions for linear regression that you are unlikely to make in a neural network problem.)

Square loss (MSE or SSE) does this. Let $y_i$ be an actual value and $\hat{y}_i$ its estimated value (prediction).

$$SSE = \sum (y_i -\hat{y}_i)^2$$

$$MSE=\dfrac{SSE}{n}$$

Except for numerical issues of doing math on a computer, these are optimized at the same parameter values of your neural network.

The squaring is critical. If a prediction is off by 1 unit, it incurs one unit of loss. If the prediction is off by 2, instead of incurring 2 units of loss, there are 4 units of loss—much worse than being off by 1. If the prediction is off by 3, wow—9 units of loss!

Square loss (MSE or SSE) does this. Let $y_i$ be an actual value and $\hat{y}_i$ its estimated value (prediction).

$$SSE = \sum (y_i -\hat{y}_i)^2$$

$$MSE=\dfrac{SSE}{n}$$

Except for numerical issues of doing math on a computer, these are optimized at the same parameter values of your neural network.

The squaring is critical. If a prediction is off by 1 unit, it incurs one unit of loss. If the prediction is off by 2, instead of incurring 2 units of loss, there are 4 units of loss—much worse than being off by 1. If the prediction is off by 3, wow—9 units of loss!

(If you look at statistics literature or some Cross Validated posts, you may see $n-p$ in the denominator of MSE, where $p$ is the number of parameters in the regression equation. This does not change the optimal value but does have some advantages in linear regression, chiefly that it is an unbiased estimate of the error variance under common assumptions for linear regression that you are unlikely to make in a neural network problem.)

Source Link
Dave
Dave
  • 4.2k
  • 1
  • 9
  • 32

Square loss (MSE or SSE) does this. Let $y_i$ be an actual value and $\hat{y}_i$ its estimated value (prediction).

$$SSE = \sum (y_i -\hat{y}_i)^2$$

$$MSE=\dfrac{SSE}{n}$$

Except for numerical issues of doing math on a computer, these are optimized at the same parameter values of your neural network.

The squaring is critical. If a prediction is off by 1 unit, it incurs one unit of loss. If the prediction is off by 2, instead of incurring 2 units of loss, there are 4 units of loss—much worse than being off by 1. If the prediction is off by 3, wow—9 units of loss!