I am working on
Decision Tree algorithm and at the end I calculate
RMSE value based on actual labels and predicted values (for regression). Now what I am having difficulty in is in understanding the significance of the
RMSE value that I get. I know that the lower the
RMSE better is the performance of the model but what
RMSE value is considered reasonable low or high? Suppose my
RMSE value comes as
20 so does that mean the performance of my model is
80% (means 80% of predictions are correct)?
I am working on
There are multiple factors to consider, but the first thing to realize is that in regression, you don't want to think about whether an example is "correct" or "incorrect" but rather how close it was to the true target value. Therefore you can ignore your original intuition about "80% of predictions are 'correct'."
Second remember that RMSE is in the same space as your target values. So it is relative to the variance in your target values. The benchmark of random guessing should get you an
RMSE = standard_deviation. So lower than this, your model is demonstrating some ability to learn; above that number, you haven't even learned to guess the mean correctly.
There isn't a cutoff for "my model is doing well" in RMSE space, just like with other metrics. Everything is relative to a naive solution/benchmark or the state-of-the-art.
The root-mean-square error (RMSE) is a frequently used measure of the differences between values (sample and population values) predicted by a model and the values actually observed. The RMSE for your training and your test sets should be very similar if you have built a good model. If the RMSE for the test set is much higher than that of the training set, it is likely that you've badly over fit the data.