1
$\begingroup$

In what way would exploratory data analysis aid in feature selection, other than to preprocess the data ? Say, if a bivariate analysis was conducted for each predictor variable w.r.t. the target variable, in what way would this help with feature selection, if possible ?

$\endgroup$

1 Answer 1

1
$\begingroup$

This is an interesting but broad question.

Imagine PCA. You yse it for exploring the data embedded in lower dimensional space but the first $n$ principale components are also used as the features (after projection of data on them).

Or you use correlation analysis and remove (deselect) features with high correlation with an existing feature.

You calculate the variance of each feature abd low variances tell you that there is no infirmation in this feature.

You inspect feature distributions according to target to determine how much they contribute to the prediction.

And of course much more ...

$\endgroup$
5
  • $\begingroup$ Or you use correlation analysis and remove (deselect) features with high correlation with an existing feature. Is it worth removing highly correlated features, if PCA were to be applied after ? You calculate the correlation/distance of feature distributions and target distribution to determine how much they contribute to the prediction Could you elaborate on this further ? Thanks $\endgroup$
    – Gale
    Mar 1, 2018 at 10:40
  • $\begingroup$ Now i see it ... talking about information theoretic concepts was better so i will update my answer but: imagine the distribution of the feature $x_i$ according to target is a mixture of two well-separated Gaussians. In a binary classification you defenitely choose that feature. But instead of correlation/distance it's better to say "relation" $\endgroup$ Mar 1, 2018 at 10:47
  • $\begingroup$ Would you then deselect features that do not appear to show any contribution to prediction, based on the distribution of that feature w.r.t. the target variable ? or is this not an appropriate approach. $\endgroup$
    – Gale
    Mar 1, 2018 at 10:57
  • $\begingroup$ That does not mean that the feature is necessarily useless i suppose. Because it may contribute in presence of another feature which you dont know yet. So only the other way around should make sense however it's my own immediate perception. I will search literature more for that and will comment here in case i found smth $\endgroup$ Mar 1, 2018 at 10:59
  • $\begingroup$ "Is it worth removing highly correlated features, if PCA were to be applied after?" ... PCA will handle them anyways $\endgroup$ Mar 1, 2018 at 11:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.