3
$\begingroup$

I successfully applied t-SNE to the number handwriting dataset. n=3823 data points (i.e. handwritten numbers) in an D=64 dimensional space (i.e. 8x8 pixels). Worked great.

Now I would like to cluster n≈60 data points in an D≈3000 dimensional space. Even after many iterations, t-SNE fairs far worse than say PCA.

Is there an upper bound on the number of dimensions (relative to the number of data points) above which applying t-SNE is not adviced?

$\endgroup$
3
  • 1
    $\begingroup$ How do you know that t-SNE "fairs far worse" than say PCA? Also t-SNE has issues with clustering in general distill.pub/2016/misread-tsne $\endgroup$ Jan 8, 2019 at 19:23
  • $\begingroup$ Brian has a good point, how you do say is doing worse? Do you have a quantitive measure in place or a 2d projection to to examine? Another suggestion is also to look at UMAP, it has shown to be faster and quite comparable in terms of identifying the manifold, see github.com/lmcinnes/umap! $\endgroup$ Jan 9, 2019 at 6:52
  • $\begingroup$ We have three groups of patients which we expect to cluster accordingly. With PCA they do reasonably well, with t-SNE they do not. Brian Spiering and Majid Mortazavi, thanks for your quick replies and very interesting links. So, very high dimensionality is no inherent problem and I will give it another go and tune parameters. $\endgroup$
    – lars20070
    Jan 9, 2019 at 8:24

2 Answers 2

1
$\begingroup$

There is no theoretical upper bound for t-SNE. However, pragmatically it will become increasingly computationally impracticable to reduce higher and higher dimensions to lower and lower dimensions. This is because t-SNE constructs a probability distribution over pairs of high-dimensional objects. In your problem, 60 data points in an 3,000 dimensional space are is more computationally intensive than 3,823 data points in 64 dimensions.

Additionally, t-SNE is a dimensionality reduction, not a clustering technique. You can directly cluster in the high-dimensional space.

$\endgroup$
1
$\begingroup$

TSNE is mainly used for used for visualising High Dimensional Data. It is not advisable to use TSNE for clustering as it preserves neither the density nor the distance. It just tries to ensure neighbours close in high dimension remain close in low dimension. But if you apply any density based or distance based clustering on the output it will not give you good results. It has been illustarted on lot of different datasets. ** Avoid Clustering on TSNE Output **

Now Regarding dimensionality, this paper clearly shows that it works better than other algorithms like ISOMAP even on Olivetti faces data set which has dimension of 92×112 = 10,304 pixels

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.