# Calculating RMSE AND R-squared from the confusion matrix

I have my confusion matrix as C.mat

8263    20    39
2       3826  14
43       7    4431


My predicted class labels are Ypred and actual labels are Ytest. Ypred size is 16000*1 and Ytest 16000*1.

I am trying to calculate the R-squared and RMSE. Is there a way to directly calculate RMSE and R-squared from the confusion matrix?

I tried this:

RMSE = sqrt(immse(Ypred, Ytest))


However, it didn't work.

I can use either R or Matlab.

The confusion matrix suggests that you are performing classification rather than regression. RMSE and R-square are measures associated with continuous variables; For categorical variables, I'd suggest using Accuracy / Recall / Precision / F1 score to measure the performance of the model.

https://www.quora.com/How-is-root-mean-square-error-RMSE-and-classification-related

• yes! i can get these accuracy recall scores from matrix but i am looking to get RMSE and R square as added metrics. Does this mean that there isnt a way to get Rsquare and RMSE values from Ypred and Ytest Dec 5, 2016 at 1:41
• Even if we use one-hot encoding to encode the 3-class variable and obtain 2 binary variables to represent the 3 groups, I don't see how we could compute the R-Square / RMSE. If you consider the error of Ypred <> Ytest = 1, then RMSE is the same as accuracy isn't it? :) Why are you after RMSE / R-square for categorical target variable?
– jkyh
Dec 5, 2016 at 1:46

In addition to what the other respondents said, I would like to add that using RMSE and MSE as metrics to evaluate a classifier can actually be a good idea if the classes are ordinal. In this case, there is a natural order between the categories, i.e. good > moderate > poor. Because missing by one class is less bad than missing by two or more, you want to use a metric that takes this into account. If you want to use only a single metric, MSE and MAE are your best choices according to Gaudette and Japkowicz (2009). Judging from their approach, RMSE could also be a good choice.

Here is one way to calculate MSE and RMSE from a confusion matrix in MATLAB:

cm = [8263   20   39
2 3826   14
43    7 4431];

se = 0;
for i = 1:3
for j = 1:3
se = se + cm(i,j) * (i-j)^2;
end
end

mse = se / sum(sum(cm));
rmse = sqrt(mse);


If you use RMSE for classification, then effectively every squared error will be a 1. The mean squared error will be your misclassification rate and the RMSE the square root of that.

The square root of the mean/average of the square of all of the error.

The use of RMSE is very common and it makes an excellent general purpose error metric for numerical predictions.

Compared to the similar Mean Absolute Error, RMSE amplifies and severely punishes large errors.

$$RMSE = \sqrt{\frac{1}{n}\sum_{i=1}^{n}((y_{i} - \hat{y}_{i})^2)}$$

MATLAB Code

RMSE = sqrt(mean((y-y_pred).^2));


R Code

RMSE <- sqrt(mean((y-y_pred)^2))


Python (Scikit Learn)

from sklearn.metrics import mean_squared_error
RMSE = mean_squared_error(y, y_pred)**0.5