I have a large data set with many features (70). By doing preprocessing (removing features with too many missing values and those that are not correlated with the binary target variable) I have arrived at 15 features. I am now using a decision tree to perform classification with respect to these 15 features and the binary target variable so I can obtain feature importance. Then, I would choose features with high importance to use as an input for my clustering algorithm. Does using feature importance in this context make any sense?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
It might make sense, but it depends what you're trying to do:
- If the goal is to predict the binary target for any instance, a classifier will perform much better.
- If the goal is to group instances by their similarity, loosely taking the binary target into account indirectly, then clustering in this way makes sense. This would correspond to a more exploratory task where the goal is to discover patterns in the data, focusing on the features which are good indicators of the target (it depends how good they actually are).
-
$\begingroup$ Erwan, I'm looking for some papers providing proof about your second statements. I've you some ideas ? I read some papers but I did not find model agnostic proof and there are quite old. ex: Feature Selection in Clustering Problems or Features selection from high-dimensional web data using clustering analysis $\endgroup$– CyrilDec 7, 2021 at 13:34
-
$\begingroup$ @Cyril Mind that this was an answer for the unusual case asked by OP about clustering when there is a known target variable. The papers you link are about the standard clustering setting with no target variable. $\endgroup$– ErwanDec 7, 2021 at 19:04