I'm training to predict a single value y (continuos in [0,1]) based on a number of variables x1, ..., x45. These are variables related to various measurements of the same value using different techniques - some of them may be irrelevant.

1) Would it make sense to use a CNN? I read that it works best for things like image recognition/classification, and this is clearly not the case

2) Which NN would work best? The benchmarks would be the naive linear regression and the average x

  • $\begingroup$ This question cannot be answered correctly without further information about the function you are trying to regress. One chosen regression model will not work best for all possible function mappings from IR to IR. It heavily depends on the underlying function. $\endgroup$ Jun 2 '20 at 22:19

Since your outcome $y$ is restricted, you should check „beta regression“ in which the outcome is „squeezed“ into the interval $ \hat{y} \in [0, 1]$. See an example in R here: http://r-statistics.co/Beta-Regression-With-R.html

To my best knowledge, there is no "off the shelf" NN equivalent of beta regression.

You could of course use a "normal" NN for regression with dense layers with a sigmoid output layer. I.e. use a normal regression setup and change that last layer to have a sigmoid output (which is between 0,1).

What could be interesting as well is a boosted beta regression: https://cran.r-project.org/web/packages/betaboost/betaboost.pdf

Beta regression is also available for GAM (generalised additive models), which are very flexible in handling non-linearities (https://stat.ethz.ch/R-manual/R-patched/library/mgcv/html/Beta.html).

Ultimately, you need to check which approach works for you given your objectives and second order restrictions. It could (probably) also be okay to use a normal boosting approach (e.g. LightGBM or Catboost) - so without a beta link function. In this case some of the $\hat{y}$ could be $>1$ or $<0$. If you do not care too much about few "overshooting" predictions, you could also achieve good results with "normal" regression.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.