Proper feature selection methods for classifying signal into two category

Question

I have a confusion to decide which feature selection method that I should employ in my research whose objective is to analyze which features that are significant in representing a certain condition of the human body, lie on two categories: normal or not.

I used multiple sensors to determine some features and plan to characterize the signal through the feature respecting the patient condition.

I have explored so many articles and blog about the fittest method for doing feature selection in classifying data into two categories. and this one is pretty good to me:

https://machinelearningmastery.com/feature-selection-with-real-and-categorical-data/

It said that we can use ANOVA and kendall's rank,

but on another site, it is mentioned that RFE could be employed too in selecting features for classifying data, another paper also said that we could employ mRmR, genetic algorithm, and Relief,

I am not sure about this, but is it mean I can employ all of those on feature selection without any further restriction according to my research objective?

Answer 1

Generally, feature selection is somewhat of a fuzzy process. Since you usually don't have a ground truth in predicting biology, you will always have to consider how realistic whatever you came up with is. I would recommend to start with the most simple method and see how your model performs. After establishing this as a baseline, you can then explore other feature selection methods to see if that improves your model.

For most models, there are general assumptions that should be met - check this before trying to optimise your features. So, for example, if one feature is just the sum of featureA and featureB, you don't need it in your data. Also, of course, always be cautious about over/under-fitting and do proper cross-validation. And if you come up with a set of features, it might make sense to think about their implications since ML is prone to find shortcuts that are only specific to your data.