Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [tsne]

t-SNE (t-distributed stochastic neighbor embedding) is a technique for dimensionality reduction.

2
votes
2answers
48 views

Grouping already clustered data (with a pre-defined x and y)

I have an already clustered data set (I wanna keep my x and y), where there's clearly a small group of elements in the middle that don't follow the expected patterns. I can select them manually, but ...
2
votes
2answers
43 views

Using t-SNE to track progress of a word vector embedding model. Pitfalls?

I've been training a word2vec/doc2vec model on a large amount of text. I recently stumbled across the t-SNE package, and am finding it wonderful at finding hidden structure in high-dimensional data. ...
2
votes
0answers
40 views

t-SNE on extremely high-dimensional spaces

I successfully applied t-SNE to the number handwriting dataset. n=3823 data points (i.e. handwritten numbers) in an D=64 dimensional space (i.e. 8x8 pixels). Worked great. Now I would like to cluster ...
0
votes
1answer
17 views

Good t-SNE or bad t-SNE?

I have used tsne to visualise a large dataset and it has produced the following graph. I need help interpreting it, as I have never seen a tsne graph like this before! I am aware that not much ...
0
votes
1answer
76 views

Reconstructing original data points from t-SNE output

I have been trying to understand t-SNE for a while now and I have this very basic question on the comparison of PCA and t-SNE, on which I would really appreciate some help. In case of PCA suppose the ...
0
votes
0answers
15 views

Visualize similar looking words plus time related feature

I would like to visualize a high dimensional space consisting of words, the way the look and when were they more used. For the similarity I use various ranges of ngrams on the letters (this ...
6
votes
2answers
162 views

What does the long curve-shape t-SNE mean?

I use 1-D CNN input 1*512 size time series data which randomly fragment segment, the output will classify input into 10 classes. After training the CNN, I apply t-SNE to the prediction which I fed in ...
1
vote
1answer
53 views

Does it make sense to visualize data with a linear relationship using tSNE?

I have used tSNE several times to visualize high dimensional data for cluster analysis, and it has always worked quite well when data falls into clusters. However, does it make any sense to use tSNE ...
0
votes
1answer
169 views

How to recreate T-SNE dimensions deterministically?

So I have a set of 3000 features from which I would like to generate clusters. I passed my features through the T-SNE algorithm to reduce dimensionality to 2 features, and clusters are really visible ...
1
vote
1answer
115 views

Is there t-SNE in WEKA?

I want to use t-SNE in WEKA just for visualization purposes. I tried to look at the package manager but could not find any implementation of it. Is there anything that I can do to achieve it?
4
votes
2answers
2k views

How to create interactive plot of thousands of images as output of t-SNE?

I have many images that I want to plot as a result of running t-SNE and I want to be able to interactively explore them. matplotlib does not allow enough interactivity to explore, and plotly is too ...
0
votes
0answers
25 views

Optimal dimensionality reduction methods for large square, mirrored, matrices (MxM)?

I have 10,000 x 10,000 (up to 100,000 x 100,000) matrices. The matrices denote pairwise similarity between elements of a same sequence. M(1,2) is the similarity between points 1 and 2. M(500,4250) is ...
1
vote
1answer
201 views

Converting similarity matrix before inputting to t-sne

I have a cosine similarity matrix where I want to adjust it to inputto t-sne. I fond the following explanation in a FAQ. As mentioned there I have made the diagonals to zero. what does it mean by <...
1
vote
1answer
366 views

Cluster doc2vec using Affinity Propagation

I want to cluster my document vectors (doc2vec) using affinity propagation. However, I am just confused if I should use cosine similarity or cosine distance to cluster my document vectors. Currently, ...
1
vote
1answer
938 views

Simple way to visualise word2vec vector space

I want to visualize my word2vec vector space (with zoom in and zoom out). I found a really interesting GitHub project named word2vec explorer. However, it does not seem to work in windows and mac. ...
2
votes
0answers
765 views

t-SNE plotting DBSCAN clustering results very scattered issue

We are trying a DBSCAN clustering model on our 30,000 samples with 15 features each. We tuned the epsilon parameter small enough to make sure the radius of the clustering circle is small while it does ...
2
votes
0answers
74 views

Dimensionality reduction with known colinearity between features

Let's say that I have sparse feature vectors and I'd like to use dimensionality reduction in order to visualize them more easily. What if I have some prior knowledge on the colinearity between my ...
7
votes
2answers
83 views

Ways to reconstruct shuffled pixels of a video file?

Suppose that you have a video file which pixel order has been shuffled once. That is, a random order have been defined once and applied to all frames. Does it exist some known approach for ...
3
votes
3answers
1k views

tsne for prediction

I have a traditional prediction setting, with a training data set train and a test data set test. I do not know the outcome <...
8
votes
1answer
1k views

Does nearest neighbour make any sense with t-SNE?

Answers on here have stated that the dimensions in t-SNE are meaningless, and that the distances between points are not a measure of similarity. However, can we say anything about a point based on it'...
0
votes
1answer
176 views

How do I calculate a similarity matrix with a Student-t kernel?

As the title says, how do I calculate a similarity matrix with an un-normalized Student-t kernel? I'm attempting to calculate Kullback-Leibler divergence for different t-SNE runs, but need a Q-matrix ...
10
votes
1answer
538 views

t-SNE: Why equal data values are visually not close?

I have 200 data points that have the same values on all features. After t-SNE dimension reduction they doesn't look so equal anymore, just like this: Why aren't they on the same point in the ...
1
vote
1answer
729 views

Retain similarity distances when using an autoencoder for dimensionality reduction

I am trying to reduce the dimensionality of topic vectors (300, 1) to a two dimensional space. This has been done with various methods (e.g. t-SNE and autoencoders). A published example of reducing ...
3
votes
1answer
414 views

What are 2D dimensionality reduction algorithms good for?

It seems to me that t-SNE and other dimensionality reduction algorithms which reduce the dimensionality to two dimensions are mainly used to get an impression of the dataset. If done well, they look ...
18
votes
1answer
852 views

Are t-sne dimensions meaningful?

Are there any meanings for the dimensions of a t-sne embedding? Like with PCA we have this sense of linearly transformed variance maximizations but for t-sne is there intuition besides just the space ...
4
votes
1answer
568 views

Given a t-SNE plot, how can I infer the “most correct” labels? How does one understand its structure?

Let's say I begin with an exceptionally large dataframe (e.g. imported/munged from tsv files). Several of these columns are categorical labels. (As a more concrete example, let's imagine a group of ...
6
votes
2answers
7k views

Clustering high dimensional data

TL;DR: Given a big image dataset (around 36 GiB of raw pixels) of unlabeled data, how can I cluster the images (based on the pixel values) without knowing the number of clusters ...
2
votes
1answer
375 views

Organize TSNE data into grid

I have some data reduced by TSNE into a 2D representation, which shows clear spatial features. However, I'd like to format this into a grid – not just snapping data to the nearest grid square but ...
4
votes
2answers
144 views

How to equalize the pairwise affinity perplexities when implementing t-SNE?

I'm trying to implement the t-SNE algorithm: I found that to compute the pairwise affinities, I have to follow this: My problem is computing $\sigma_i$. In the Wikipedia I found: The bandwidth of ...
4
votes
2answers
153 views

Is t-SNE just for visualization?

I have used the t-SNE algorithm to visualize my high dimensional data. However, I was wondering if this is a practical method for inference?
5
votes
2answers
3k views

How to calculate KL-divergence between matrices

Given there are two matrices of dimensionality 100x2 with absolute values ranging from -50 to +50. Is it possible to determine the kl-divergence by applying the entropy algorithm from scipy.stats to ...
4
votes
1answer
193 views

Can I apply Clustering algorithms to the result of Manifold Visualization Methods?

Some methods related to manifold-learning are commonly stated as good-for-visualization, such as T-SNE and self-organizing-maps (SOM). I understand that when referring specifically to "visualization" ...
11
votes
1answer
1k views

Can closer points be considered more similar in T-SNE visualization?

I understand from Hinton's paper that T-SNE does a good job in keeping local similarities and a decent job in preserving global structure (clusterization). However I'm not clear if points appearing ...