# Questions tagged [tsne]

t-SNE (t-distributed stochastic neighbor embedding) is a technique for dimensionality reduction.

48 questions
Filter by
Sorted by
Tagged with
1answer
80 views

### What Clustering Method Should I Use?

My data is a group of 10 thousand points (each having an node location (x,y)) that are spread across a plane. They are also chromatically-colored based on their weight. I need to finalize a bayesian ...
2answers
190 views

### Grouping already clustered data (with a pre-defined x and y)

I have an already clustered data set (I wanna keep my x and y), where there's clearly a small group of elements in the middle that don't follow the expected patterns. I can select them manually, but ...
2answers
181 views

### Ways to reconstruct shuffled pixels of a video file?

Suppose that you have a video file which pixel order has been shuffled once. That is, a random order have been defined once and applied to all frames. Does it exist some known approach for ...
2answers
84 views

### If in t-SNE digaram of binary classification both classes follow the similar curve what does t-SNE diagram show?

If in t-SNE digaram of binary classification both classes follow the similar curve what does t-SNE diagram show for instance: Figure1 or Figure2
1answer
84 views

### How to reduce position changes after dimensionality reduction?

Disclaimer: I'm a machine learning beginner. I'm working on visualizing high dimensional data (text as tdidf vectors) into the 2D-space. My goal is to label/modify those data points and recomputing ...
1answer
57 views

### what information can we obtain from t-SNE?

I see that t-SNE can help us reduce dimensions and visualize the data. But what information are we gaining from this visualization? As we know that the new axis don't have a meaning in our context. ...
1answer
289 views

### How do I calculate a similarity matrix with a Student-t kernel?

As the title says, how do I calculate a similarity matrix with an un-normalized Student-t kernel? I'm attempting to calculate Kullback-Leibler divergence for different t-SNE runs, but need a Q-matrix ...
4answers
577 views

### Is t-SNE just for visualization?

I have used the t-SNE algorithm to visualize my high dimensional data. However, I was wondering if this is a practical method for inference?
0answers
13 views

### Why is a 2D->2D tSNE transformation not idempotent (up to random influences)?

My understanding of tSNE is that probability distributions are optimized in a way to maintain Euclidean (by default) distances (on average) when transforming from the input to the output space. If ...
2answers
63 views

### can I use t-sne or PCA to reduce number of classes?

I wanted to know if I can use t-sne or PCA to reduce the number of classes depending on the similarity between them. For example, if I have 100 classes of 100 different animals and would like to put ...
0answers
59 views

### How to plot centroids and clusters resulting from a KMean model based on a text variable

I hope you can help as after several attempts, I'm no longer sure I can get a decent result. I have a text corpus made of several documents, like the one below (which has been simplified for the sake ...
1answer
312 views

### comparison of t-SNE and PCA and truncate SVD

How to compare the trucate SVD ,PCA, and T-SNE? What we can say about features if t-SNE and PCA and truncate SVD digaram is in this figure?
2answers
1k views

### python tsne.transform does not exist?

I am trying to transform two datasets: x_train and x_test using tsne. I assume the way to do this is to fit tsne to x_train, and then transform x_test and x_train. But, I am not able to transform any ...
3answers
1k views

### What does the long curve-shape t-SNE mean?

I use 1-D CNN input 1*512 size time series data which randomly fragment segment, the output will classify input into 10 classes. After training the CNN, I apply t-SNE to the prediction which I fed in ...
0answers
93 views

### (UMAP) Why does umap.transform() put new data in the periphery of old clusters?

UMAP is a dimensionality reduction method like t-SNE and PCA. Unlike t-SNE, UMAP can incorporate new data and do a forward transform on the data. While experimenting, I made an interesting observation ...
1answer
286 views

### Using TSNE to Visualize Clusters in Python

I'm using TSNE to visualize my clusters but the output seems a bit strange. There are supposed to be 3 clusters but instead, there are 4 lines. Is there something wrong with how I'm visualizing them ...
1answer
246 views

### Meaning of curved line shape distribution in t-SNE plot

I have blood test results with 20 features from 170 patients, and trying to predict a categorical disease outcome. The input features are all continuous values except sex (0:male, 1:female) There are ...
1answer
134 views

### Dimensionality reduction with prior knowledge of colinearity between features

Let's say that I have sparse feature vectors and I'd like to use dimensionality reduction in order to visualize them more easily. Dimensionality reduction techniques like PCA will estimate ...
1answer
145 views

### long shape line in tSNE plot

My question is similar to this post: What does the long curve-shape t-SNE mean? but the problem is that my data is not time series, but I also get these long shape line in tSNE plot . I don't know ...
0answers
19 views

### Suggestions on non-linear dimensionality reduction for small, one-hot encoded dataset

I wish to apply non-linear dimensionality reduction on a very small dataset (less than 100 observations). The dataset is very sparse, of approx 20 columns, each containing either 0 or 1. It's the ...
2answers
9k views

### Clustering high dimensional data

TL;DR: Given a big image dataset (around 36 GiB of raw pixels) of unlabeled data, how can I cluster the images (based on the pixel values) without knowing the number of clusters ...
1answer
131 views

### Why is PCA often used before t-sne for problems when the goal is only to reduce dimensionality?

Ex: Matlab's t-sne tutorials frequently use PCA https://www.mathworks.com/help/stats/tsne-settings.html " Process Data Using t-SNE Obtain two-dimensional analogs of the data clusters using t-SNE. ...
0answers
123 views

### Approximating t-SNE embeddings for out-of-sample data

I have a large amount of data which has been reduced to two dimensions using t-SNE. Additional data points keep arriving, which I would like two-dimensional embeddings for, but this cannot be achieved ...
0answers
55 views

### Is there a representation of the separating hyperplane in t-sne?

I have used t-sne to visualize a set of images which I have used for training a binary classifier. Let us assume that the binary classifier is trained to detect cat(1) vs. no-cat(0). I have used the ...
2answers
656 views

### Using t-SNE to track progress of a word vector embedding model. Pitfalls?

I've been training a word2vec/doc2vec model on a large amount of text. I recently stumbled across the t-SNE package, and am finding it wonderful at finding hidden structure in high-dimensional data. ...
0answers
150 views

### t-SNE on extremely high-dimensional spaces

I successfully applied t-SNE to the number handwriting dataset. n=3823 data points (i.e. handwritten numbers) in an D=64 dimensional space (i.e. 8x8 pixels). Worked great. Now I would like to cluster ...
1answer
408 views

### Good t-SNE or bad t-SNE?

I have used tsne to visualise a large dataset and it has produced the following graph. I need help interpreting it, as I have never seen a tsne graph like this before! I am aware that not much ...
1answer
960 views

### Reconstructing original data points from t-SNE output

I have been trying to understand t-SNE for a while now and I have this very basic question on the comparison of PCA and t-SNE, on which I would really appreciate some help. In case of PCA suppose the ...
1answer
519 views

### Cluster doc2vec using Affinity Propagation

I want to cluster my document vectors (doc2vec) using affinity propagation. However, I am just confused if I should use cosine similarity or cosine distance to cluster my document vectors. Currently, ...
1answer
274 views

### Does it make sense to visualize data with a linear relationship using tSNE?

I have used tSNE several times to visualize high dimensional data for cluster analysis, and it has always worked quite well when data falls into clusters. However, does it make any sense to use tSNE ...
2answers
5k views

### How to create interactive plot of thousands of images as output of t-SNE?

I have many images that I want to plot as a result of running t-SNE and I want to be able to interactively explore them. matplotlib does not allow enough interactivity to explore, and plotly is too ...
1answer
971 views

### How to recreate T-SNE dimensions deterministically?

So I have a set of 3000 features from which I would like to generate clusters. I passed my features through the T-SNE algorithm to reduce dimensionality to 2 features, and clusters are really visible ...
1answer
320 views

### Is there t-SNE in WEKA?

I want to use t-SNE in WEKA just for visualization purposes. I tried to look at the package manager but could not find any implementation of it. Is there anything that I can do to achieve it?
1answer
440 views

### Converting similarity matrix before inputting to t-sne

I have a cosine similarity matrix where I want to adjust it to inputto t-sne. I fond the following explanation in a FAQ. As mentioned there I have made the diagonals to zero. what does it mean by <...
1answer
1k views

### Simple way to visualise word2vec vector space

I want to visualize my word2vec vector space (with zoom in and zoom out). I found a really interesting GitHub project named word2vec explorer. However, it does not seem to work in windows and mac. ...
3answers
5k views

### tsne for prediction

I have a traditional prediction setting, with a training data set train and a test data set test. I do not know the outcome <...
0answers
1k views

### t-SNE plotting DBSCAN clustering results very scattered issue

We are trying a DBSCAN clustering model on our 30,000 samples with 15 features each. We tuned the epsilon parameter small enough to make sure the radius of the clustering circle is small while it does ...
1answer
4k views

### Does nearest neighbour make any sense with t-SNE?

Answers on here have stated that the dimensions in t-SNE are meaningless, and that the distances between points are not a measure of similarity. However, can we say anything about a point based on it'...
1answer
1k views

### t-SNE: Why equal data values are visually not close?

I have 200 data points that have the same values on all features. After t-SNE dimension reduction they doesn't look so equal anymore, just like this: Why aren't they on the same point in the ...
1answer
308 views

### Can I apply Clustering algorithms to the result of Manifold Visualization Methods?

Some methods related to manifold-learning are commonly stated as good-for-visualization, such as T-SNE and self-organizing-maps (SOM). I understand that when referring specifically to "visualization" ...
1answer
1k views

### Retain similarity distances when using an autoencoder for dimensionality reduction

I am trying to reduce the dimensionality of topic vectors (300, 1) to a two dimensional space. This has been done with various methods (e.g. t-SNE and autoencoders). A published example of reducing ...
1answer
2k views

### Are t-sne dimensions meaningful?

Are there any meanings for the dimensions of a t-sne embedding? Like with PCA we have this sense of linearly transformed variance maximizations but for t-sne is there intuition besides just the space ...
1answer
546 views

### What are 2D dimensionality reduction algorithms good for?

It seems to me that t-SNE and other dimensionality reduction algorithms which reduce the dimensionality to two dimensions are mainly used to get an impression of the dataset. If done well, they look ...
1answer
1k views

### Given a t-SNE plot, how can I infer the “most correct” labels? How does one understand its structure?

Let's say I begin with an exceptionally large dataframe (e.g. imported/munged from tsv files). Several of these columns are categorical labels. (As a more concrete example, let's imagine a group of ...
1answer
645 views

### Organize TSNE data into grid

I have some data reduced by TSNE into a 2D representation, which shows clear spatial features. However, I'd like to format this into a grid – not just snapping data to the nearest grid square but ...
2answers
277 views

### How to equalize the pairwise affinity perplexities when implementing t-SNE?

I'm trying to implement the t-SNE algorithm: I found that to compute the pairwise affinities, I have to follow this: My problem is computing $\sigma_i$. In the Wikipedia I found: The bandwidth of ...
1answer
3k views

### Can closer points be considered more similar in T-SNE visualization?

I understand from Hinton's paper that T-SNE does a good job in keeping local similarities and a decent job in preserving global structure (clusterization). However I'm not clear if points appearing ...
2answers
7k views

### How to calculate KL-divergence between matrices

Given there are two matrices of dimensionality 100x2 with absolute values ranging from -50 to +50. Is it possible to determine the kl-divergence by applying the entropy algorithm from scipy.stats to ...