In linear regression, why we generally use RMSE instead of MSE.

The rational I know is its easy to minimize the error in RMSE instead of MSE by Gredient Decent.

But need to know exact reason.

  • 1
    $\begingroup$ How do you figure we use RMSE instead of MSE? They’re equivalent loss functions, by the way, save some numerical goofiness on a computer. Anyway, the advantage of RMSE is that it’s in the same units as the response variable. MSE and SSE are in squared units. $\endgroup$ – Dave Jan 19 at 16:24

However Root Mean Square Error seems similar to MSE and is the root of it, gradient of RMSE with respect to $i^{th}$ prediction differes from that of MSE.

$$\frac{\sigma{RMSE}}{\sigma{y_i}} =\frac{1}{2}\frac{1}{\sqrt{MSE}}\frac{\sigma MSE}{\sigma y_i}$$

Gradient of RMSE equals to the gradient of MSE multiplied by this $\frac{1}{2}\frac{1}{\sqrt{MSE}}$ value which is constant and is called learning rate. And it shows that RMSE and MSE cannot be interchangeably used when using gradient based methods like when it comes to use linear regression with gradient descent optimization algorithm.

Further, in cases when it is better to give more weight (higher error) to large errors, RMSE can be a better error measure. I believe it depends on your data distribution which one of them pick to use. But in linear regression, maybe not caring about the points which are considerably off the actual prediction line, lead to being biased to considering these points and having a bad model. So, it might be better to give a higher error value to these points.

This might also help and this.

| improve this answer | |

Two reasons, mainly the second I think.

  1. The RMSE is an indication of the noise levels in the scale of standard deviations.
  2. The RMSE has nice mathematical properties for fast calculations. (Its gradient is linear and propagates easily)
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.