clustering before or after PCA?

Question

I'm newbie into data science, and I had some problems dealing with my project. I'm trying to visualize multidimensional data into 2D after clustering with using a lot of methods. (kmeans, DBSCAN, OPTICS, agglomerative, spectral...)

I have multidimensional data. (11 columns - attributes , 150K rows - number of data). It is slightly sparse-alike data, for example, which means one datum has numeric values like (0, 0, 6.5, 0, 0, 7.5, 0, 0, 4.5, 0, 0)

So, each datum has approximately 2~5 non-zero attribute values...

Below is not exactly same with my project, but it's similar.

https://scikit-learn.org/stable/auto_examples/cluster/plot_cluster_comparison.html

But, as I'm new to this, I am curious about the sequence of PCA and Clustering. I think there are 2 scenarios.

[1. Do clustering before PCA]

That means, I am using PCA just for visualization. But I have a question. In that case, If I process clustering with raw data, are all clustering algorithm (mentioned above) fit to my data type well.

[2. Do clustering after PCA]

In this case, I have other problems. My data's importance of components are like below.

                          PC1    PC2    PC3    PC4     PC5     PC6     PC7    PC8     PC9    PC10
Standard deviation     1.4173 1.1836 1.1141 1.0108 0.99109 0.95231 0.89091 0.8456 0.71542 0.64610
Proportion of Variance 0.2009 0.1401 0.1241 0.1022 0.09823 0.09069 0.07937 0.0715 0.05118 0.04174
Cumulative Proportion  0.2009 0.3410 0.4651 0.5673 0.66551 0.75620 0.83558 0.9071 0.95826 1.00000

As far as I've understood about visualizing multivariate data into 2D, I have to choose 2 PCs.(e.g.> PC1, PC2). However, my data's PV is slightly lower than I'd expected.

So, is it okay if I choose (PC1, PC2) to coordinates to be clustered and process clustering? Also, can I choose other PCs (e.g. PC5, PC8) to coordinates to be clustered?

Answer 1

1. This is often done to visualize if there is any structure in the data. Often you color the clustering differently to check if samples from the same cluster are close.

2. Often data contains a lot of redundant information. With many dimensions, you get the curse of dimensionality. This can lead to a few large and many small clusters. By reducing the dimensionality to a few informative features the clustering solution often improves. This is very dependent on the dataset. Perhaps all of your dimensions are informative then dimensionality reduction won't help much. Look at scoring metrics such as silhouette analysis/score, gap statistic, or elbow method.

In your case, most of the dimensions seem informative. Perhaps you can remove one or two dimensions, but it likely won't have much effect on the clustering solution.

You can choose whatever PC's you'd like but they are ordered according to the amount of variance they explain. You can look at how much the different original features contribute to the different PC's.