# Coefficient of determination is close to 1 but the value of RMSE is large. What does it mean?

I am working with the DecisionTreeRegressor and trying to understand how well the data fits the model. I calculated both RMSE and coefficient of determination. At a certain depth, coefficient of determination has the value equal to 0.9918744073066561 but the value of RMSE is equal to 75.0025. I cannot understand this. The value of RMSE is quite large but the value of the coefficient of determination is close to 1.0. What does it really mean? Is the model/fit good enough?

from sklearn.tree import DecisionTreeRegressor
from sklearn.metrics import mean_squared_error

def rmse(y_true, y_pred):
return math.sqrt(mean_squared_error(y_true, y_pred))

sample_depth = np.linspace(1,40, num = 40, dtype=int)
dt_score_list = []
for index, depth in enumerate(sample_depth):
boosted_regressor.fit(X_train, y_train)
dt_score_list.append(boosted_regressor.score(X_test, y_test))
print(boosted_regressor.score(X_test, y_test), rmse(y_test, boosted_regressor.predict(X_test)), depth)

• RMSE is based entirely on your data, if you have a lot of data you can expect a lot of variation coming from a lot of points (which can be small but it does add up). Sep 25, 2018 at 13:20

Compared to $$R^2$$ $$RMSE$$ is dependent on the variance of values. Even if $$R^2$$ is close to 1 but the standard deviation is high the value of $$RMSE$$ will also be high. $$RMSE=\sigma_{y_{true}}\sqrt{1-R^2}$$ where $$\sigma$$ is a standard deviation

As can be seen from the formula $$RMSE$$ and $$R^2$$ have a strict mathematical relationship that shows that $$RMSE$$ changes according to standard deviation ($$\sigma$$) and the higher the standard deviation of the true values the higher the value of $$RMSE$$.

$$RMSE$$ shows the deviation of the error in units of the values, while $$R^2$$ shows the share of variance that is explained by the model. You can explain 99% of the variance ($$R^2$$) but your numbers are in millions of dollars and your error still varies in thousands of dollars ($$RMSE$$) Whether thousands of dollars is ok for your error or not depends on the case. If you predict profit plus or minus 1000\$is ok. If you predict revenue then plus or minus 1000\$ could be a difference between a profitable and a non-profitable company.

Proof of the formula:

According to sklearn the formula for $$R^2$$ is

$$R^2 = 1 - \frac{\sum_i^n(y_{true} - y_{pred})^2}{\sum_i^n(y_{true} - \overline{y}_{true})^2}$$

RMSE is \begin{align} RMSE &= {\sqrt{\frac1n\sum_i^n(y_{true} - y_{pred})^2}} \\RMSE^2 &=\frac1n\sum_i^n(y_{true} - y_{pred})^2 \\\frac{RMSE^2}{\frac1n\sum_i^n(y_{true} - \overline{y}_{true})^2} &=\frac{\frac1n\sum_i^n(y_{true} - y_{pred})^2}{\frac1n\sum_i^n(y_{true} - \overline{y}_{true})^2} \\\frac{RMSE^2}{\frac1n\sum_i^n(y_{true} - \overline{y}_{true})^2} &=\frac{\sum_i^n(y_{true} - y_{pred})^2}{\sum_i^n(y_{true} - \overline{y}_{true})^2} \\1-\frac{RMSE^2}{\frac1n\sum_i^n(y_{true} - \overline{y}_{true})^2} &=1-\frac{\sum_i^n(y_{true} - y_{pred})^2}{\sum_i^n(y_{true} - \overline{y}_{true})^2} \\1-\frac{RMSE^2}{\frac1n\sum_i^n(y_{true} - \overline{y}_{true})^2} &=R^2\\1-\frac{RMSE^2}{\sigma_{y_{true}}^2} &=R^2 \\RMSE&=\sigma_{y_{true}}\sqrt{1-R^2} \end{align}

where $$\sigma$$ is a standard deviation and $$\overline{y}$$ is an average