I'm newbie into data science, and I had some problems dealing with my project. I'm trying to visualize multidimensional data into 2D after clustering with using a lot of methods. (kmeans, DBSCAN, OPTICS, agglomerative, spectral...)

I have multidimensional data. (11 columns - attributes , 150K rows - number of data). It is slightly sparse-alike data, for example, which means one datum has numeric values like (0, 0, 6.5, 0, 0, 7.5, 0, 0, 4.5, 0, 0)

So, each datum has approximately 2~5 non-zero attribute values...

Below is not exactly same with my project, but it's similar.


But, as I'm new to this, I am curious about the sequence of PCA and Clustering. I think there are 2 scenarios.

[1. Do clustering before PCA]

That means, I am using PCA just for visualization. But I have a question. In that case, If I process clustering with raw data, are all clustering algorithm (mentioned above) fit to my data type well.

[2. Do clustering after PCA]

In this case, I have other problems. My data's importance of components are like below.

                          PC1    PC2    PC3    PC4     PC5     PC6     PC7    PC8     PC9    PC10
Standard deviation     1.4173 1.1836 1.1141 1.0108 0.99109 0.95231 0.89091 0.8456 0.71542 0.64610
Proportion of Variance 0.2009 0.1401 0.1241 0.1022 0.09823 0.09069 0.07937 0.0715 0.05118 0.04174
Cumulative Proportion  0.2009 0.3410 0.4651 0.5673 0.66551 0.75620 0.83558 0.9071 0.95826 1.00000

As far as I've understood about visualizing multivariate data into 2D, I have to choose 2 PCs.(e.g.> PC1, PC2). However, my data's PV is slightly lower than I'd expected.

So, is it okay if I choose (PC1, PC2) to coordinates to be clustered and process clustering? Also, can I choose other PCs (e.g. PC5, PC8) to coordinates to be clustered?

  • $\begingroup$ If you have many values that are zero, you might not get good results from the clustering methods. Imagine you have just two dimensions and most object (rows) have one 0 somewhere. Many objects would end up on one of the axis in your real coordinate space. Is this what you want to cluster? The same problem (but in multiple dimensions) applies in your case. You might have to think, whether you might define a metric that takes this into account and then apply clustering using this metric. The metric gives you are "similarity matrix" and you get use this for clustering (see your own link). $\endgroup$
    – Make42
    Feb 1, 2020 at 11:55

1 Answer 1

  1. This is often done to visualize if there is any structure in the data. Often you color the clustering differently to check if samples from the same cluster are close.

  2. Often data contains a lot of redundant information. With many dimensions, you get the curse of dimensionality. This can lead to a few large and many small clusters. By reducing the dimensionality to a few informative features the clustering solution often improves. This is very dependent on the dataset. Perhaps all of your dimensions are informative then dimensionality reduction won't help much. Look at scoring metrics such as silhouette analysis/score, gap statistic, or elbow method.

In your case, most of the dimensions seem informative. Perhaps you can remove one or two dimensions, but it likely won't have much effect on the clustering solution.

You can choose whatever PC's you'd like but they are ordered according to the amount of variance they explain. You can look at how much the different original features contribute to the different PC's.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.