Return to Answer

The dimensions of the low dimensional space have no meaning. Note that the t-SNE loss function is soley based on the distances between points ($y_i$ and $y_j$) and probability distributions over those distances ($p_{ij}$ and $q_{ij}$):

$$ \frac{\delta C}{\delta y_i}=4 \sum_j(p_{ij} - q_{ij})(y_i-y_j)(1+||y_i -y_j||^2)^{-1} $$

Thus there is no projection from the whole high-dimensional space to the low-dimensional space, t-SNE only finds a mapping from a specific set of high-dimensional points to a specific set of low dimensional points. Because there is no function from one space to the other there is also no inherent meaning of the axes.

Things you can imagine to illustrate this:

• Rotating or translating the high-dimensional or low-dimensional space does not influences distances between the points. Hence, t-SNE does not care about rotation or translation in both spaces. Thus there is not absolute interpretation of the axes.
• The t-Student distribution has fat tales. This causes the low-dimensional representation to be invariant to changes in points that are far away in the high-dimensional space. This also causes that points that are far away in the high-dimensional space can be either reasonably far away, far away or really far away in the low dimensional space. In this sense it stretches certain parts of the low-dimensional axes (in any arbitrary direction).

That being said, t-SNE is a dimension reduction technique suitable for visualization. Not for clustering, feature extraction or feature selection.

Also: the paper.

The dimensions of the low dimensional space have no meaning. Note that the t-SNE loss function is solely based on the distances between points ($y_i$ and $y_j$) and probability distributions over those distances ($p_{ij}$ and $q_{ij}$):

$$ \frac{\delta C}{\delta y_i}=4 \sum_j(p_{ij} - q_{ij})(y_i-y_j)(1+||y_i -y_j||^2)^{-1} $$

Thus there is no projection from the whole high-dimensional space to the low-dimensional space, t-SNE only finds a mapping from a specific set of high-dimensional points to a specific set of low dimensional points. Because there is no function from one space to the other there is also no inherent meaning of the axes.

Things you can imagine to illustrate this:

• Rotating or translating the high-dimensional or low-dimensional space does not influences distances between the points. Hence, t-SNE does not care about rotation or translation in both spaces. Thus there is not absolute interpretation of the axes.
• The t-Student distribution has fat tails. This causes the low-dimensional representation to be invariant to changes in points that are far away in the high-dimensional space. This also causes that points that are far away in the high-dimensional space can be either reasonably far away, far away or really far away in the low dimensional space. In this sense it stretches certain parts of the low-dimensional axes (in any arbitrary direction).

That being said, t-SNE is primarily a visualization technique and its dimension reduction effectiveness for other purpose is not obvious (probably not suitable for clustering, feature extraction or feature selection).

Also: the paper.

The dimensions of the low dimensional space have no meaning. Imagine rotating the whole low-dimensional map, the pair-wise distances do not change and consequently the pair-wise probabilities remain equal. Likewise, rotating the high-dimensional map does not change their pair-wise probabilities. Hence, rotating either does not change the quality of the solution.

This illustrates that the t-SNE loss function is soley based on the distances between points. Furthermore, it only defines a mapping from a specific set of high-dimensional points to a set of low dimensional points. That's the core of the answer. t-SNE does not define a function that defines a relation between the high-dimensional space (instead of points) and the low-dimensional space, it only defines a mapping between the points. It finds this mapping by looking at the (gaussian) distances between points in the high-dimensional map and the (t-Student) distances in the low-dimensional map.

That being said, t-SNE is a dimension reduction technique suitable for visualization. Not for clustering, feature extraction or feature selection.

Also: the paper.

The dimensions of the low dimensional space have no meaning. Note that the t-SNE loss function is soley based on the distances between points ($y_i$ and $y_j$) and probability distributions over those distances ($p_{ij}$ and $q_{ij}$):

$$ \frac{\delta C}{\delta y_i}=4 \sum_j(p_{ij} - q_{ij})(y_i-y_j)(1+||y_i -y_j||^2)^{-1} $$

Thus there is no projection from the whole high-dimensional space to the low-dimensional space, t-SNE only finds a mapping from a specific set of high-dimensional points to a specific set of low dimensional points. Because there is no function from one space to the other there is also no inherent meaning of the axes.

Things you can imagine to illustrate this:

• Rotating or translating the high-dimensional or low-dimensional space does not influences distances between the points. Hence, t-SNE does not care about rotation or translation in both spaces. Thus there is not absolute interpretation of the axes.
• The t-Student distribution has fat tales. This causes the low-dimensional representation to be invariant to changes in points that are far away in the high-dimensional space. This also causes that points that are far away in the high-dimensional space can be either reasonably far away, far away or really far away in the low dimensional space. In this sense it stretches certain parts of the low-dimensional axes (in any arbitrary direction).

That being said, t-SNE is a dimension reduction technique suitable for visualization. Not for clustering, feature extraction or feature selection.

Also: the paper.

The dimensions of the low dimensional space have no meaning. Imagine rotating the whole low-dimensional map, the pair-wise distances do not change and consequently the pair-wise probabilities remain equal. Likewise, rotating the high-dimensional map does not change their pair-wise probabilities. Hence, rotating either does not change the quality of the solution.

This illustrates that the t-SNE loss function is soley based on the distances between points. Furthermore, it only defines a mapping from a specific set of high-dimensional points to a set of low dimensional points. That's the core of the answer. t-SNE does not define a function that defines a relation between the high-dimensional space (instead of points) and the low-dimensional space, it only defines a mapping between the points. It finds this mapping by looking at the (gaussian) distances between points in the high-dimensional map and the (t-Student) distances in the low-dimensional map.

Also: the paper.

The dimensions of the low dimensional space have no meaning. Imagine rotating the whole low-dimensional map, the pair-wise distances do not change and consequently the pair-wise probabilities remain equal. Likewise, rotating the high-dimensional map does not change their pair-wise probabilities. Hence, rotating either does not change the quality of the solution.

This illustrates that the t-SNE loss function is soley based on the distances between points. Furthermore, it only defines a mapping from a specific set of high-dimensional points to a set of low dimensional points. That's the core of the answer. t-SNE does not define a function that defines a relation between the high-dimensional space (instead of points) and the low-dimensional space, it only defines a mapping between the points. It finds this mapping by looking at the (gaussian) distances between points in the high-dimensional map and the (t-Student) distances in the low-dimensional map.

That being said, t-SNE is a dimension reduction technique suitable for visualization. Not for clustering, feature extraction or feature selection.

Also: the paper.