Timeline for Does nearest neighbour make any sense with t-SNE?
Current License: CC BY-SA 3.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 28 at 16:48 | comment | added | PSub | I agree with the qualtitative interpretation as being the ranking of similar pairs being preserved in the embedded space compared to the original. However, I'm not quite sure what you mean when you say that the KL divergence should be zero? Do you mean between the embedded and original space. That would only be the case assuming the dimension reduction is optimal. | |
| May 27 at 15:59 | comment | added | Antonios Sarikas | @PSub Can we interpret the distances in the embedded space conceptually? I mean the goal of t-SNE is to preserve pairwise similarities in the original space. That is, if $p_{i1} > p_{i2}$ in the original space, then $y_{i1} > y_{i2}$ in the embedded space. Of course, preserving the rankings of the similarities doesn't necessarily preserve the original distances, but at least would let us interpret qualitatively a t-SNE plot. Closer points in the embedded space should be closer also in the original one. Of course, this might also require the KL divergence to be 0 (after optimization). | |
| Sep 18, 2020 at 17:12 | comment | added | nvergos |
Thank you for this answer and the code snippet PSub. I am wondering whether this can be edited to accommodate for unsupervised settings where we don't have any information about the ground truth, and we rely on the accurate embedding by SNE. If we fit TSNE for 2 components, could we then train kNN on (X_embedded[0], X_embedded[1]), and then when we have new unseen data, first call TSNE.transform to embed into trained embedding and then call predict() to get the position of the new data point wrt the original map?
|
|
| Aug 29, 2017 at 4:24 | comment | added | PSub | Yes, perplexity is one of the major factors which affects how close points stay to each other. Early exaggeration, intuitively is how tight clusters in the original space and how much space there will be between them in the embedded space (so it's a mixture of both perplexity and early exaggeration which affects the distances between points. Regarding your last question, the answer is yes, this is because of the exponentiation of the norm, which could cause issues in the embedding space, so there is a chance of misclassification. | |
| Aug 28, 2017 at 13:59 | history | bounty ended | geometrikal | ||
| Aug 28, 2017 at 6:40 | vote | accept | geometrikal | ||
| Aug 28, 2017 at 6:40 | comment | added | geometrikal | Thanks for the good answer. To summarize: Points that have a high similarity have a high probability of staying close. I'm guessing that the perplexity parameter controls how many points are used for the probability calculation, so clusters can become disjoint if perplexity is low. Can you comment on early exaggeration? Also, I assume the probability of points being outliers or misclassified (having all their NN in another class) using the TSNE space, would be increased if they are consistent after multiple TSNE with random initialisation? | |
| Aug 24, 2017 at 16:41 | history | edited | PSub | CC BY-SA 3.0 |
added 249 characters in body
|
| Aug 22, 2017 at 20:09 | history | answered | PSub | CC BY-SA 3.0 |