1
$\begingroup$

Does the appliance of R-squared to non-linear models depends on how we calculate it? $R^2 = \frac{SS_{exp}}{SS_{tot}}$ is going to be an inadequate measure for non-linear models since an increase of $SS_{exp}$ doesn't necessarily mean that the variance is decreasing, but if we calculate it as $R^2 = 1 - \frac{SS_{res}}{SS_{tot}}$, then it's as much meaningful for non-linear models as it is for linear ones. I asked a similar question here where I showed that R-squared is no worse for non-linear models

So, what is the particular reason to say that in the case of non-linear models $\mathbf{R^2}$ loses its interpretation of proportion of variance explained? In both cases (I mean linear and non-linear models) we learn by how much your model's variance decreased with respect to its initial (total) variance. If $R^2 = 0.86 \text %$ then your model's variance decreased by $0.86 \text %$ (no matter whether it's linear or not).

EDIT:

  1. $SS_{tot} = \|y - \bar y\|^2$
  2. $SS_{exp} = \|\hat y - \bar y\|^2$
  3. $SS_{res} = \|y - \hat y\|^2$.

Where:

  1. $y$ is a vector of true answers;
  2. $\bar y$ is a vector whose elements are mean of $y$;
  3. $\hat y$ is a vector with our model's predictions.
Improve this question
$\endgroup$
9
  • $\begingroup$ What is $SS_{exp}?$ $\endgroup$
    – Dave
    Oct 11, 2021 at 23:48
  • $\begingroup$ @Dave, Hello! I've edited. $\endgroup$
    – mathgeek
    Oct 12, 2021 at 0:33
  • $\begingroup$ In a linear model, is $SS_{tot}=SS_{res}+SS_{exp}?$ What about in a nonlinear model? stats.stackexchange.com/q/427390/247274 stats.stackexchange.com/questions/494274/… $\endgroup$
    – Dave
    Oct 12, 2021 at 0:38
  • $\begingroup$ @Dave, Yes, I see that. But do you actually calculate $SS_{exp}$ when you use $R^2$? No. The formula you use is $R^2 = 1 - \frac{SS_{res}}{SS_{tot}}$ which doesn't care about $SS_{exp}$ at all. It just calculate the difference $SS_{tot} - SS_{res}$ in the nominator and after that divides it by $SS_{tot}$. Why is it worse for nonlinear models, then? We still learn by how much your model's variance decreased with respect to its initial (total) variance. $\endgroup$
    – mathgeek
    Oct 12, 2021 at 0:47
  • $\begingroup$ Doesn’t the “exp” mean “explained”? $\endgroup$
    – Dave
    Oct 12, 2021 at 0:48

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.