Hierarchical Clustering and Variable Selection

Question

I am using "Single linkage" hierarchical algorithm to cluster my data points with Gower Distance as my data have both qualitative and quantitative variables.

After applying this for the full model (all variables) I would like to start excluding those variables which are actually the not so important for my data. I was thinking of using principal component analysis (PCA) but I can't because my variables are a mixture of both categorical and continuous. Can someone suggest what is best method to select variables?

Finally I would like to use the Elbow Method to check exactly what is the optimal number of clusters?

Can someone help me with this logic?

I am using R-Studio for my analysis.

Answer 1

PCA is only suitable for continuous variables, and sensitive to scaling. So don't use it here.

Instead, you meat want to look at classic information measures whether some attribute is "correlated" with the clusters. For example Gini, mutual information etc.

Similarly, the Elbow method is a bad idea. It never was a good idea in the first place because of scaling... But it's really only suitable for choosing the k in k-means if you do many random restarts and look at the best partitionings found. But even then: there is no mathematical definition of the "elbow". Most of the time, there is no elbow, and people just pick some random k this way because they don't consider this option...nit's probably widely misused. But it certainly must not be used with hierarchical clustering!