1
$\begingroup$

  • If we have high number of features (more than 50), should we use T-SNE ?

  • According to https://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html:

    It is highly recommended to use another dimensionality reduction method (e.g. PCA for dense data or TruncatedSVD for sparse data) to reduce the number of dimensions to a reasonable amount (e.g. 50) if the number of features is very high. This will suppress some noise and speed up the computation of pairwise distances between samples

  1. It seems that if we have more than 50 features, it is better to work with PCA and not with T-SNE, Did I understand it correctly ?
  2. Why T-SNE is not good with high number of features ?
  3. Why the document suggest to work with PCA and not with other dimension-reduction (like UMAP) ?
Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ t-SNE works well with much more than 50 features. In NLP research, it is usual to see it applied to hundreds of features. However, in general, UMAP is better than t-SNE for any purpose, at least in my experience; probably UMAP is not mentioned in the t-SNE docs because they were written before its existence. $\endgroup$
    – noe
    Jun 22, 2021 at 7:03

2 Answers 2

Reset to default
1
$\begingroup$

Be aware that PCA is a linear dimensional reduction algorithm, whereas t-SNE or UMAP are non linear (=gaussian). Consequently the results are usually better with t-SNE or UMAP, even with a large number of features. Then, you should be carefull with the features because not all of them have the same weight, and some are too noisy, which creates bad results (no clear clusters). I usually recommend using less features or simplified data, see if the results are correct, and then increase the number of features or the data complexity. Then, the main advantage of UMAP is that clusters are correlated to each other, but t-SNE could be better in correlations point to point. Note that t-SNE could require data normalisation, whereas UMAP doesn't.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thanks. Why the documentation recommends to work with PCA if the number of features is higher ? $\endgroup$
    – user3668129
    Jun 22, 2021 at 10:24
  • 1
    $\begingroup$ PCA is usefull to detect which are the most important features. The documentation recommends to use PCA in order to reduce the numbers of features taking the 50 most important ones, and then you would be able to use t-SNE or UMAP effectively. But if you have already a reasonable amount of features (3 to ~100), you don't need to apply PCA with truncated SVD. Note that you can't have a clear dimensionality reduction with PCA: you will mainly see features correlations scores, but very often without clear clusters. $\endgroup$
    – Nicolas Martin
    Jun 22, 2021 at 10:32
  • $\begingroup$ Thanks. If we use PCA as first step (to reduce the numbers of features to the 50 most important ones) and TSNE as second-step, than, why not to use PCA only (and select the number of features we want to work with) ? $\endgroup$
    – user3668129
    Jun 22, 2021 at 10:42
  • $\begingroup$ PCA is great to have a rough estimation of features correlations, whereas t-SNE is able to differentiate precisely rows by projecting all their n features (=n dimensions) in a 2D plane or a 3D space. youtu.be/wvsE8jm1GzE $\endgroup$
    – Nicolas Martin
    Jun 22, 2021 at 11:22
  • $\begingroup$ You can play by comparing dimensional reduction results with PCA, UMAP or t-SNE using this web page: projector.tensorflow.org $\endgroup$
    – Nicolas Martin
    Jun 22, 2021 at 11:59
1
$\begingroup$

I have never seen sound analysis about suitableness of using PCA to pre-processing data for t-SNE. All those suggestions were just based on few examples. On the other side, I have encountered many cases (for data with >20K features), that PCA significantly alters the embedding, especially when large perplexity was used or correlation was used as distance metric. So, normally I won't use PCA for pre-processing for t-SNE.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.