1
$\begingroup$

I'm working on a simple linear regression model to predict 'Label' based on 'feature'. The two variables seems to be highly correlate corr=0.99. After splitting the data sample for to training and testing sets. I make predictions and evaluate the model.

metrics.mean_squared_error(Label_test,Label_Predicted) = 99.17777494521019
metrics.r2_score(Label_test,Label_Predicted) = 0.9909449021176512

Based on the r2_score my model is performing perfectly. 1 being the highest possible value. But when it comes to the mean squared error, I don't know if it shows that my model is performing well or not.

  1. How can I interpret MSE here ?

  2. If I had multiple algorithms and the same data sets, after computing MSE or RMSE for all models, how can I tell which one is better in describing the data ?

  3. R2 score is 0.99, is this suspicious ? Or expected since the label and feature are highly correlated?

          Feature        Label
0   56171.757812    56180.234375
1   56352.500000    56363.476562
2   56312.539062    56310.859375
3   56432.539062    56437.460938
4   56190.859375    56199.882812
...     ...     ...
 24897  56476.484375    56470.742188
 24898  56432.148438    56432.968750
 24899  56410.312500    56428.437500
 24900  56541.093750    56541.015625
 24901  56491.289062    56499.843750

enter image description here

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Whether you model is performing well or not depends on your business case, you might hive tiny RMSE or great looking score on whatever metric you are using, but it just not enough to solve the business problem, in that case the model is not performing well.

  1. MSE is just that Mean Squared Error

  2. Both MSE and RMSE measure by how much the predicted result deviates from actual, because of the squared term more weight is given to larger errors, and because of square root in RMSE, it is in the same units as dependent variable. MAE, Mean Absolute Error is another useful metric to look at when you are evaluating a regression model; it is also easier to interpret.

  3. Given your data, R-squared seems fine to me.

Improve this answer
$\endgroup$
2
  • $\begingroup$ thanks for your answer. MSE is in this case is 99. How can I interpret that? I mean I know that the best r2 score is 1. But for the MSE should I compare it for example to the Standart deviation of the data sample? How can I tell that MSE being 99 is good or bad? $\endgroup$
    – the phoenix
    Commented Mar 9, 2021 at 4:34
  • $\begingroup$ @thephoenix, since RMSE is in the units of the dependent variable, you could divide it by mean of the dependent variable, and it will give you a unitless measure. Your idea of comparing with Standart deviation of the data sample seems reasonable too. The point that I want to stress is that these kind of metrics don't tell the full story and this is true for r-squared as well, whether the model is good or not depends on whether it can solve the business problem it is faced with. $\endgroup$
    – Akavall
    Commented Mar 9, 2021 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.