1
$\begingroup$

The Data look something like this:

[0, ...., 0, 2.8, 10, 4.5,]

So there are a lot of zeros to predict. And just a few non zeroes.

Apllying R2 to a potentially predictive model for this kind of Data is not feaseable: The difference between all zero prediction and the expected values is not that big.

Is there a R2-like metric to aid this situation ?

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ What goes wrong when you use $R^2$ despite your reservations? You’re still seeing how much variability you’re regression accounts for compared to always guessing the mean of the response variable. $\endgroup$
    – Dave
    Apr 2, 2020 at 11:28
  • $\begingroup$ I run my model with simple R^2 , yet the obtained R2 fails to convey an improvement while tuning hyperparameters. $\endgroup$
    – Alexander Vocaet
    Apr 2, 2020 at 12:52
  • $\begingroup$ So your model is doing a poor job of predicting. That’s an issue with your model, not $R^2$. $\endgroup$
    – Dave
    Apr 2, 2020 at 13:03
  • $\begingroup$ that why i want to tune the model into a better predicting model through hyperparameters. But for this, the metric for model evaluation has to convey the right message. R2 failed for me in this case $\endgroup$
    – Alexander Vocaet
    Apr 2, 2020 at 16:39
  • $\begingroup$ Why is $R^2$ failing you by (seemingly correctly) reporting a poor model fit? $\endgroup$
    – Dave
    Apr 2, 2020 at 16:44

1 Answer 1

Reset to default
1
$\begingroup$

Looks like your data are severely unbalanced if they are almost all zeros. Try data balancing or some form of cross-validation. To your original point: there is not a simple R^2 metric fix.

Improve this answer
$\endgroup$
3
  • $\begingroup$ This is a Model with multiple Targets, each has their non-zero values on different variables. Balancing is only possible in by filtering the prediction. Currently, I will approach this by filtering the none-zero Data. Filling the Amount of none-zero with otherwise similar data, yet 0 in specific target. Thus obtaining a 50/50 balancing for R2 calculation $\endgroup$
    – Alexander Vocaet
    Apr 2, 2020 at 12:59
  • $\begingroup$ It sounds like you're going to invent new data for data balancing, rather than sampling over existing data? That sounds like a spectacularly bad idea. I guess I didn't understand your comment. $\endgroup$
    – Dave Kielpinski
    Apr 2, 2020 at 18:29
  • $\begingroup$ no, sry i did not convey it right. i am not creating new data. I sample some extra Data from samples that are simliar to the ones with the non-zero values. $\endgroup$
    – Alexander Vocaet
    Apr 2, 2020 at 22:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.