16
$\begingroup$

I know that there is no a clear answer for this question, but let's suppose that I have a huge neural network, with a lot of data and I want to add a new feature in input. The "best" way would be to test the network with the new feature and see the results, but is there a method to test if the feature IS UNLIKELY helpful? Like correlation measures (http://www3.nd.edu/~mclark19/learn/CorrelationComparison.pdf) etc?

$\endgroup$
  • 1
    $\begingroup$ A non-random correlation might be an indicator that the feature is useful. But I'm not so sure about pre-training tests that could rule ideas out. The paper you link makes it clear that non-linear correlations are not well detected by the available tests, but a neural net has a chance of finding and using them. $\endgroup$ – Neil Slater Jul 10 '14 at 11:28
16
$\begingroup$

A very strong correlation between the new feature and an existing feature is a fairly good sign that the new feature provides little new information. A low correlation between the new feature and existing features is likely preferable.

A strong linear correlation between the new feature and the predicted variable is an good sign that a new feature will be valuable, but the absence of a high correlation is not necessary a sign of a poor feature, because neural networks are not restricted to linear combinations of variables.

If the new feature was manually constructed from a combination of existing features, consider leaving it out. The beauty of neural networks is that little feature engineering and preprocessing is required -- features are instead learned by intermediate layers. Whenever possible, prefer learning features to engineering them.

$\endgroup$
  • $\begingroup$ I always thought to compare the value to predict with the features, you are talking about correlation between features. Is your answer applicable also to my case? in theory I should add only new features that are correlated to the value to predict, right? $\endgroup$ – marcodena Jul 10 '14 at 19:06
  • $\begingroup$ That's also a valuable metric -- just updated my answer to address that as well. $\endgroup$ – Madison May Jul 10 '14 at 19:18
  • $\begingroup$ In short, strong correlations with the value to predict is a great sign, but weak correlation with the value to predict is not necessarily a bad sign. $\endgroup$ – Madison May Jul 10 '14 at 19:19
  • $\begingroup$ Thanks. I'm writing a report and I wanted to show the linear/non-linear correlations in order to justify the features (even before the results). Does it make any sense? From your answer I could make a matrix of correlations but maybe it's nosense $\endgroup$ – marcodena Jul 10 '14 at 19:27
  • 1
    $\begingroup$ I would use non-linear correlations, but ok thanks $\endgroup$ – marcodena Jul 10 '14 at 23:17
0
$\begingroup$

If you are using sklearn, there is a good function available called model.feature_importances_. Give it a try with your model/new feature and see if it helps. Also look here and here for examples.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.