I'm comparing the performance of several models on the same data using cross-validation (holding out 1/n of the data as a test set, fitting the model on the remaining data, testing on the test set). I then average out-of-sample performance across the folds and select the model which performs best. The question is, how should I decide between using average MSE or average R2?

Intuitively, it originally made more sense to me to average MSE rather than R2 to obtain the overall model performance. MSE is a mean, so in a loose sense the average is itself an MSE. But R2 is averaged in this way more frequently in literature I've seen. Even though for a single fit R2 and MSE are monotonic with respect to each other, the averaging can lead to a situation where the model with lowest average MSE is different from the model with the highest average R2. So the choice does matter.

Is anyone able to clarify how I should decide which to use, or whether some other metric/aggregation method is better?

I'm not looking for generic "MSE and R2 are designed to measure different things" answers. I'm comparing models on the same data, so absent this averaging, the choice wouldn't affect which model I'd choose. I'm interested in how to decide which to average as I'm doing here. Thanks!


1 Answer 1


There are differences between these two metrics that we must consider, but I think you should not choose just one metric. Instead, you should use multiple metrics and check whether these metrics are consistent with each other. What I mean is, that if the MSE is low and R² is high, it is beneficial to understand why. If they are not consistent, finding the reasons could help you understand the dataset and the machine learning model better.

To address your question:

If your data has varying scales, R² might be a better choice because it normalizes performance regardless of the scale of the target variable. This can be particularly useful when comparing models across different datasets or response variables.

MSE is more sensitive to outliers due to the squaring of errors. R², being a relative measure, might provide a more balanced view if outliers are present in your data.

Averaging R² can be misleading because it does not linearly relate to model performance. The average R² might not represent the true predictive capability of the model across different folds.

Averaging MSE, on the other hand, directly reflects the model’s average performance in terms of error magnitude.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.