1
$\begingroup$

I am using forest fire dataset and applied neural network model. I tried to generate REC curve, this is how it looks like. Pretty weird!!!

Neural network I have also applied XGBoost but the REC curve is almost parallel to X-axis. I don't understand how to make sense out of it?
I also want to understand how to interpret REC curve in general?

$\endgroup$

1 Answer 1

0
$\begingroup$

The title of the graph says "REC curve for various models" but this curve is for a single model. It shows as Y the percentage of "correct" predictions by the model depending on what "correct" means, which is given as X: for example if correct means an error less than 10 ha (on X axis), then the model is correct around 55% of the time (on Y axis). In other words the graph splits the instances between correct/incorrect ones based on whether the error value (difference between true and predicted value) is lower than a threshold X.

In particular this graph shows that:

  • this model predicts very few values with an error less than 8
  • this model very often (around 50%) predicts values which are 8-9 ha off the true value.
  • there are couple more values that the model predicts with an error around 15.
  • more than 40% of the predicted values are not visible on this graph, which means that their error is higher than 20.

I'd suggest plotting the true vs. predicted value, it's easier to interpret (although it can be harder to see where are the large groups of instances).

$\endgroup$
4
  • $\begingroup$ I have removed the title of the image to avoid confusion. One way is to plot true vs predicted value as you suggested. Can you name some other popular ways to evaluate regression model graphically? $\endgroup$ Commented May 6, 2021 at 22:22
  • $\begingroup$ Well one can plot the distribution of errors for instance. there are probably other ways, depends what one wants to analyze. $\endgroup$
    – Erwan
    Commented May 6, 2021 at 22:28
  • $\begingroup$ The same neural network model predicts output with rmse value of approximately 15. But as you said "there are couple more values that the model predicts with an error around 15." Why is it so? RMSE value should not be that good IMO. $\endgroup$ Commented May 6, 2021 at 22:32
  • $\begingroup$ First RMSE is an error value, so lower is better. The RMSE around 15 is explained by the fact that there are approximately 50% instances with error around 8-9 and more than 40% with error higher than 20 (not visible on the graph which is cut at 20). $\endgroup$
    – Erwan
    Commented May 7, 2021 at 9:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.