# How to choose the features for a neural network?

I know that there is no a clear answer for this question, but let's suppose that I have a huge neural network, with a lot of data and I want to add a new feature in input. The "best" way would be to test the network with the new feature and see the results, but is there a method to test if the feature IS UNLIKELY helpful? Like correlation measures etc?

• A non-random correlation might be an indicator that the feature is useful. But I'm not so sure about pre-training tests that could rule ideas out. The paper you link makes it clear that non-linear correlations are not well detected by the available tests, but a neural net has a chance of finding and using them. Jul 10 '14 at 11:28

A very strong correlation between the new feature and an existing feature is a fairly good sign that the new feature provides little new information. A low correlation between the new feature and existing features is likely preferable.

A strong linear correlation between the new feature and the predicted variable is an good sign that a new feature will be valuable, but the absence of a high correlation is not necessary a sign of a poor feature, because neural networks are not restricted to linear combinations of variables.

If the new feature was manually constructed from a combination of existing features, consider leaving it out. The beauty of neural networks is that little feature engineering and preprocessing is required -- features are instead learned by intermediate layers. Whenever possible, prefer learning features to engineering them.

• I always thought to compare the value to predict with the features, you are talking about correlation between features. Is your answer applicable also to my case? in theory I should add only new features that are correlated to the value to predict, right? Jul 10 '14 at 19:06
• That's also a valuable metric -- just updated my answer to address that as well. Jul 10 '14 at 19:18
• In short, strong correlations with the value to predict is a great sign, but weak correlation with the value to predict is not necessarily a bad sign. Jul 10 '14 at 19:19
• Thanks. I'm writing a report and I wanted to show the linear/non-linear correlations in order to justify the features (even before the results). Does it make any sense? From your answer I could make a matrix of correlations but maybe it's nosense Jul 10 '14 at 19:27
• I would use non-linear correlations, but ok thanks Jul 10 '14 at 23:17

If you are using scikit-learn, there is a good function available called model.feature_importances_.

Give it a try with your model/new feature and see if it helps. Also look here and here for examples.