I have used tSNE several times to visualize high dimensional data for cluster analysis, and it has always worked quite well when data falls into clusters.

However, does it make any sense to use tSNE to visualize the independent variables in a dataset which has a linear relationship $y = X \theta + b$, where $y$ is the target value and $X$ are the independent variables (b is the bias)?

As I understand tSNE, it is used more for capturing local structures in high dimensional data, so it is probably not suited for visualizing data for linear regression - is this a wrong assumption?

  • $\begingroup$ If you know the relationship is linear (or that the relationship is that simple in 2D space in general), why do want to visualise it using t-SNE? Do you have enough features to actually separate the variables into clusters? $\endgroup$
    – n1k31t4
    Jun 21 '18 at 8:43
  • $\begingroup$ @n1k31t4 it can be linear in more than 2 dimensions (not just 2D!) :) $\endgroup$
    – pcko1
    Jun 21 '18 at 8:47
  • 1
    $\begingroup$ Of course, apologies - I jumped to conclusions seeing your example equation! $\endgroup$
    – n1k31t4
    Jun 21 '18 at 10:09

Your assumption is right, the results are in general misleading.

Suppose your (linearly correlated) data have missing points in some range:

3-d plot of linearly correlated data

Than for t-SNE the two subset of data will be two different clusters, even if they lie on the same linear distribution:

t-SNE transformated data

But, if you are actually interested in the fact that those two structures are separated, then t-SNE is a good choice to visualize it.

So a proper answer should be: it depends on what you need.


Here the code used for this example:

import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from sklearn.manifold import TSNE

def function(x, y):
    return 4+3*x + 4*y

X=np.concatenate([x1, x2])

Z=function(X, Y)

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

ax.scatter(X, Y, Z, c='r', marker='o')

for x, y, z in zip (X, Y, Z):
    data.append([x, y, z])

data_embedded = TSNE(n_components=2).fit_transform(data)

plt.scatter([x for x, y in data_embedded], [y for x, y in data_embedded], color='r')

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.