2
$\begingroup$

I'm trying to do a correlation analysis between inputs and outputs inspecting the data in order to understand which input variables to include. What could be a threshold in the correlation value to consider a variable eligible to be an input for my Neural Network?

$\endgroup$
2
$\begingroup$

Given non-linearity of neural networks, I believe correlation analysis isn't a good way to estimate importance of variables. For example, imagine that you have 2 input variables - x1 and x2 - and following conditions hold:

  • cor(x2, y) = 1 if x1 = 1
  • cor(x2, y) = 0 otherwise
  • x1 = 1 in 10% of cases

That is, x2 is a very good predictor for y, but only given that x1 = 1, which is the case only in 10% of data. Taking into account correlations of x1 and x2 separately won't expose this dependency, and you will most likely drop out both variables.

There are other ways to perform feature selection, however. Simplest one is to train your model with all possible sets of variables and check the best subset. This is pretty inefficient with many variables, though, so many ways to improve it exist. For a good introduction in best subset selection see chapter 6.1 of Introduction to Statistical Learning.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.