In linear regression, why we generally use RMSE instead of MSE? The rationale I know is that it's easy to minimize the error in RMSE instead of MSE by Gradient Descent, but I need to know the exact reason.

  • 3
    $\begingroup$ How do you figure we use RMSE instead of MSE? They’re equivalent loss functions, by the way, save some numerical goofiness on a computer. Anyway, the advantage of RMSE is that it’s in the same units as the response variable. MSE and SSE are in squared units. $\endgroup$ – Dave Jan 19 '20 at 16:24

However RMSE seems similar to MSE and is the root of it, gradient of RMSE with respect to $i^{th}$ prediction differs from that of MSE.

$$\frac{\sigma{RMSE}}{\sigma{y_i}} =\frac{1}{2}\frac{1}{\sqrt{MSE}}\frac{\sigma MSE}{\sigma y_i}$$

Gradient of RMSE is equal to the gradient of MSE multiplied by this $\frac{1}{2}\frac{1}{\sqrt{MSE}}$ value which is constant and is called learning rate. And it shows that RMSE and MSE cannot be interchangeably used when using gradient based methods like when it comes to use linear regression with gradient descent optimization algorithm.

Further, in cases when it is better to give more weight (higher error) to large errors, RMSE can be a better error measure. I believe it depends on your data distribution which one of them to pick to use. But in linear regression, maybe not caring about the points which are considerably off the actual prediction line leads to being biased to considering these points and having a bad model. So, it might be better to give a higher error value to these points.

This and this might also help.


Two reasons, mainly the second one, I think.

  1. The RMSE is an indication of the noise levels in the scale of standard deviations.
  2. The RMSE has nice mathematical properties for fast calculations (Its gradient is linear and propagates easily).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.