Are t-sne dimensions meaningful?

Question

Are there any meanings for the dimensions of a t-sne embedding? Like with PCA we have this sense of linearly transformed variance maximizations but for t-sne is there intuition besides just the space we define for mapping and minimization of the KL-distance?

Answer 1

The dimensions of the low dimensional space have no meaning. Note that the t-SNE loss function is solely based on the distances between points ($y_i$ and $y_j$) and probability distributions over those distances ($p_{ij}$ and $q_{ij}$):

$$ \frac{\delta C}{\delta y_i}=4 \sum_j(p_{ij} - q_{ij})(y_i-y_j)(1+||y_i -y_j||^2)^{-1} $$

Thus there is no projection from the whole high-dimensional space to the low-dimensional space, t-SNE only finds a mapping from a specific set of high-dimensional points to a specific set of low dimensional points. Because there is no function from one space to the other there is also no inherent meaning of the axes.

Things you can imagine to illustrate this:

• Rotating or translating the high-dimensional or low-dimensional space does not influences distances between the points. Hence, t-SNE does not care about rotation or translation in both spaces. Thus there is not absolute interpretation of the axes.
• The t-Student distribution has fat tails. This causes the low-dimensional representation to be invariant to changes in points that are far away in the high-dimensional space. This also causes that points that are far away in the high-dimensional space can be either reasonably far away, far away or really far away in the low dimensional space. In this sense it stretches certain parts of the low-dimensional axes (in any arbitrary direction).

That being said, t-SNE is primarily a visualization technique and its dimension reduction effectiveness for other purpose is not obvious (probably not suitable for clustering, feature extraction or feature selection).

Also: the paper.