0
$\begingroup$

I have used the $R^2$ metric to determine how well my neural network performs a non-linear regression. And it seems to work. The plots look almost identical, and I get an $R^2$ value of 0.93... it seems to work perfectly, however I have been told that $R^2$ should only work for linear data. Is this true or false?

$\endgroup$
  • $\begingroup$ Yes, R-squared measures relative improvement in MSE. Make sure you are using a clean validation strategy. $\endgroup$ – Michael M Nov 2 at 20:58
  • $\begingroup$ @MichaelM please could you elaborate on what you mean by a clean validatuon strategy? I am currently using K-fold cross validation, with k=10 on a dataset of roughly 5 million data points $\endgroup$ – DeepLearner Nov 3 at 14:05
-1
$\begingroup$

The $R^2$ calculation for nonlinear regression models can very a lot depending on the software / application / etc. Also, they involve a lot of approximations in order to make any calculation and are not as easily interpretable as the linear case. There are many other methods that will give you a good and very interpretable result directly from your data. For example, some kind of cross-validation like K-fold cross-validation or bootstrap methods give you a good sense of how trustworthy your estimates are directly from your data without relying on a bunch of normal approximations. Lastly, for most applications I would also recommend using some goodness of fit measure that takes into account parsimony like the AIC or BIC.

If it's a linear regression (linear in the parameters), then yes the $R^2$ is fine and is interpretable in the commonly thought of way.

$R^2$ is good for cases where you have no model for the noise in your measurement. A good way to think of it is just as a measure of the fraction of the variance in the measurement that's covered by your fit model. But it's not necessarily what you want for cases where you're concerned you might be overfitting your data (i.e. using a fit function with more parameters than you can justify) or where the error bars on different data points are very different from one another (as happens, for example, when Poisson noise dominates). Chi2, and especially reduced chi2 (i.e. chi2 per degree of freedom) are probably more appropriate for these cases.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.