In my models, R2 in training and test sets are close to each other, but in RMSE, MSE, MAE of some models, these are very different? what is the reason Is there a solution?

  • $\begingroup$ Welcome to Data Science! How are you calculating $R^2?$ $\endgroup$
    – Dave
    Commented May 26 at 21:36
  • $\begingroup$ Thank you! I used merics.r2_score $\endgroup$ Commented May 27 at 22:31
  • $\begingroup$ In what sense are the RMSE and MAE bad but the $R^2$ is good? $\endgroup$
    – Dave
    Commented May 27 at 23:44
  • $\begingroup$ Actually, there is a big difference between RMSE in training and test set! Both R2 in test and training are 0.99, but in other metrics the difference is big. which I think shows an overfitting. $\endgroup$ Commented May 28 at 4:14
  • $\begingroup$ I argue here that sklearn.metrics.r2_score is a nonsensical calculation when it comes to out-of-sample evaluation. Do your in-sample and out-of-sample means differ considerably? (Might you have some kind of data drift?) $\endgroup$
    – Dave
    Commented May 28 at 7:59


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.