0
$\begingroup$

I have made a Random Forest model having RMSE of 0.189 and R2-Score of 0.734. How can I use statistics to measure the predictive performance of my model?

In my use case, I have to answer these questions:

  1. How well does your model work?
  2. How do you know for sure that’s how well it works?
  3. What stats did you use to prove its predictive performance and why?

Can anyone tell me how can I answer 2nd and 3rd questions?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

When you look at these numbers individually for regression it does not makes a lot of sense. For example if R Square of your model 0.734 is good or bad depends on the benchmark and problem you are trying to solve.

Benchmark Model

For first 2 question, we always try to have a benchmark model. A model which is very simple or a model which was being used as previous model and you try to improve on it. For example if you predict everything to mean if it gives your R square of say 0.75 then your model is not good though its R-sqaure is around 0.734% but if mean prediction gives you only 0.1 then you have very good model.

So you should always have benchmark to know how well a model is doing.

Evaluation Metrics

You should always choose a evaluation metrics which is aligned with your business objective and try to get very good value of it. For example if i want to predict price of a car a mean abosolute error of 50$-250$ can be good but 1000$ may not be tolerable.

Visualisation Scatter Plot

In Case of regression it is always good to overlay Predictions and actual values on a scatter plot to create a intuition how good a model is

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.