I have dataset of 50000 values (rows) and 1000 variables (columns). Since this is high dimensional, I am unable to work with just DBSCAN. So I am trying to use PCA (principle component analysis). Since PCA reduces dimension to few 100's. I request you to help you to help me with EITHER 1. Other high dimensional clustering algorithm 2. How to interpret results of PCA

  • 1
    $\begingroup$ I think it is best to just read the wiki article for PCA: en.wikipedia.org/wiki/Principal_component_analysis. $\endgroup$ – JahKnows Mar 28 '17 at 18:24
  • $\begingroup$ DBSCAN schould work on the full dimensional data, too. PCA will usually only give you little benefit if you use it just to reduce dimensionality. And with 1000 variables, 50k may be too few instances to get a reliable PCA result. I'd usually recommend at least 3*p² instances, I.e. 3 million, to get a stable PCA. $\endgroup$ – Has QUIT--Anony-Mousse Mar 28 '17 at 19:36
  • $\begingroup$ Thanks Anony-Mousse, Actually I dont know how to judge the clustering results...meaning, If its 2D we can judge which variables belong to which clusters easily from visualization BUT since my dataset has 1000Dimension, I dont now how to visualize in order to make use of clustering resulta from DBSCAN... $\endgroup$ – Abhishek Mar 29 '17 at 9:14

In short PCA, returns an orthogonal set of basis features that best represent the variance in the data.

Intuitively, imagine you want to identify whether we are talking about a dog or a cat. Your features are: size, weight, color, fur type, etc... but you also have features like weather, owner name, etc...

It should be evident that the first set of features obviously explains the variance between a dog and a cat much better than the second set. Thus, you should only consider those and completely disregard the second set.

This is what PCA does however, it adds the extra restriction that all features must be orthogonal. It then assigns a metric to each component based on the amount of variance that feature explains. You can then sort the components and extract the top $n$ components (feature reduction).

| improve this answer | |
  • $\begingroup$ the formulation might be a bit confusing for some people because it makes it sound like PCA only detects unrelated features and then returns a subset of original features. also, PCA does not reduce the dimensionality for you, at least not automatically.., $\endgroup$ – oW_ Mar 28 '17 at 20:16
  • $\begingroup$ The definition in the first sentence takes into consideration all your comments. However, you are right I should be more clear that you can then rank them and select the top $n$ components. $\endgroup$ – JahKnows Mar 28 '17 at 20:51
  • $\begingroup$ To clarify further, the features that are generated by PCA will almost never be a subset of the original features. If you're looking for a feature reduction technique which has more intuitive explanatory power, try using information entropy-based techniques. $\endgroup$ – liangjy Mar 29 '17 at 1:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.