I have 90k points in $\mathbb{R}^{32}$ (i.e., a 90k by 32 real matrix) which I want to visualize. I know I can cluster my points (k-means &c), but I want to select a few "interesting" 2-planes in $\mathbb{R}^{32}$, project the points there and scatter-plot them.

How do I select the interesting 2-planes to project to?

  • $\begingroup$ how do you define interesting? If it is activity then you might look for a membrane of maximum entropy or such. $\endgroup$ – EngrStudent Oct 16 '15 at 3:40

There are a few interesting plots and transformations you could start with, each dependent upon the purpose of your analysis. Below are some first steps I might take.

  1. If simply visually looking for clusters:
    If clusters are what you are after, then I would recommend applying a Principal Component Analysis to the dataset and then plotting the dataset with the first 2 principal components as the axes. However, the major downside to PCA is that you will have to go "unpack" the principal components to find the original variables. In other words, you'll be may be able to identify cool clusters, but it will be a little harder to tie the findings back to your 32 variables.

  2. If looking for quick relationships between variables to build on:
    With "only" 32 variables you could do a pairwise plot. However, a smarter way may be to first identify the relationships mathematically (e.g. correlation) and then plotting those variables.

  3. If looking for quick relationships in each variable to build on: Or, look at 32 histograms to start off with. Look out for clear bi-modal (or more) to begin piecing together an understanding of how your variables may contribute to an unsupervised model. If you end up look at 32 unimodal histograms then you can conclude early that no matter how you cluster, you will simply end up with a blob.

Actually, in usual analytics workflows I would go 3 > 2 > 1. But if I was fishing for clusters and just wanted to see if clusters will appear or not, PCA would be a good shortcut.

Also, feel free to sample your dataset. 90k points on your screen will likely do more harm than good.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.