# MAE estimation for k-fold cross-validation

I have code that estimates RMSE for k-fold cross-validation and I think it is correct (from book: Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition by Aurélien Géron)

scores = cross_val_score(forest_reg, a, b, scoring="neg_mean_squared_error", cv=10)
print(pd.Series(np.sqrt(-scores)).describe())


So what about MAE? Should I use (with sqrt):

scores = cross_val_score(forest_reg, a, b, scoring="neg_mean_absolute_error", cv=10)
print(pd.Series(np.sqrt(-scores)).describe())


or this (without sqrt):

scores = cross_val_score(forest_reg, a, b, scoring="neg_mean_absolute_error", cv=10)
print(pd.Series(-scores).describe())


Also for MAE estimation, it should be -scores or scores?

• For your second question, why do you take -scores for RMSE? My instinct is to do the same for MAE, but I don’t understand why you’re winding up with the wrong sign in the RMSE calculation.
– Dave
Commented May 2, 2020 at 0:56
• @Dave from book "Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition by Aurélien Géron": Scikit-Learn’s cross-validation features expect a utility function (greater is better) rather than a cost function (lower is better), so the scoring function is actually the opposite of the MSE (i.e., a negative value), which is why the preceding code computes -scores before calculating the square root. So I am pretty sure RMSE should have -scores. I think MAE should have it too, but I just wanted to reconfirm science I am new to Machine learning. Commented May 2, 2020 at 1:05
• I didn’t know that about sklearn. I wonder why they did it that way since it’s so common to talk about loss, but with that the case then you should do -scores for MAE, yes.
– Dave
Commented May 2, 2020 at 1:07

You could take the square root of MAE, but then you’d wind up with measurements with units of $$\sqrt{\}$$ or the square root of whatever units you’re using. The result is that your measure of dispersion is not in the original units, which a probably what you want.