I am working with a dataset with hundreds of features. I wish to create a simple machine learning model using 7-10 features from the original dataset. My question is this:
What quantitative metrics can I use to determine that a feature will be useful to the learning model?
I have been comparing the distribution of the target mean over the feature groups, to the target mean of the dataset as a whole. For example, take a binary feature X and a binary target. Let's say the target has a mean of 0.10 when taken over the entire dataset.
To analyze the feature X, I take the target mean for each group within feature X.
mean (X=0) = 0.07
mean (X=1) = 1.15
In this way, I can observe the effect of a feature on the target.
I know that there must be some stronger metrics which people use to determine the strength of a feature. In school I used p tests to determine the statistical significance of a variable. Is there an analog in DS/ML?