0
$\begingroup$

In a problem of an epidemiology dataset, is it desirable to keep the features that have almost constant values? For example, In case of the feature, type_of_residence Large for 97 percent and Small for 2.7 percent of subjects. Is it okay to keep this feature?

My target variable is outcome of patients and this data set is unbalanced. Like oversampling and under sampling techniques in class imbalance problem, is there any process in ML to address other predicting features?

I am not doing feature selection using ML at this point for knowing the importance of this feature. But would like to know is there any general rules regarding keeping or not keeping these kind of features in dataset.

$\endgroup$
1
$\begingroup$

If a feature is truly constant across all the instances, then it's useless for classification and it can be removed.

But in any other case, as in your example, a feature can be useful even if constant with most instances. In your example it might turn out that the target class is strongly correlated with type_of_residence, so it would be a mistake to get rid of it since your model can take advantage of it.

Unless you have a good reason to get rid of a feature (expert knowledge), keep all of them. If you must reduce the number of features (for instance to avoid overfitting), then the selection should be based on information about which features are the least informative (e.g. using some correlation measure).

Ideally the best way to know which features are useful is to try to train a classifier and evaluate it with every possible subset of features (but it's rarely possible in practice)

$\endgroup$
2
  • $\begingroup$ "In your example it might turn out that the target class is strongly correlated with type_of_residence, so it would be a mistake to get rid of it since your model can take advantage of it.". Your Wrong.. if a variable is strongly correlated with target variable then its not a contributor to the model and should be removed. $\endgroup$
    – mnm
    Jan 6 at 1:16
  • 1
    $\begingroup$ @mnm I don't understand what you mean: why do you say that a variable which is strongly correlated with the target is not a contributor? I think it is. $\endgroup$
    – Erwan
    Jan 6 at 1:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.