Questions tagged [tsne]

t-SNE (t-distributed stochastic neighbor embedding) is a technique for dimensionality reduction.

Filter by
Sorted by
Tagged with
0
votes
2answers
35 views

can I use t-sne or PCA to reduce number of classes?

I wanted to know if I can use t-sne or PCA to reduce the number of classes depending on the similarity between them. For example, if I have 100 classes of 100 different animals and would like to put ...
2
votes
1answer
53 views

Using TSNE to Visualize Clusters in Python

I'm using TSNE to visualize my clusters but the output seems a bit strange. There are supposed to be 3 clusters but instead, there are 4 lines. Is there something wrong with how I'm visualizing them ...
0
votes
1answer
44 views

comparison of t-SNE and PCA and truncate SVD

How to compare the trucate SVD ,PCA, and T-SNE? What we can say about features if t-SNE and PCA and truncate SVD digaram is in this figure?
0
votes
2answers
18 views

If in t-SNE digaram of binary classification both classes follow the similar curve what does t-SNE diagram show?

If in t-SNE digaram of binary classification both classes follow the similar curve what does t-SNE diagram show for instance: Figure1 or Figure2
1
vote
1answer
102 views

python tsne.transform does not exist?

I am trying to transform two datasets: x_train and x_test using tsne. I assume the way to do this is to fit tsne to x_train, and then transform x_test and x_train. But, I am not able to transform any ...
2
votes
1answer
39 views

long shape line in tSNE plot

My question is similar to this post: What does the long curve-shape t-SNE mean? but the problem is that my data is not time series, but I also get these long shape line in tSNE plot . I don't know ...
0
votes
1answer
50 views

Meaning of curved line shape distribution in t-SNE plot

I have blood test results with 20 features from 170 patients, and trying to predict a categorical disease outcome. The input features are all continuous values except sex (0:male, 1:female) There are ...
0
votes
0answers
16 views

Suggestions on non-linear dimensionality reduction for small, one-hot encoded dataset

I wish to apply non-linear dimensionality reduction on a very small dataset (less than 100 observations). The dataset is very sparse, of approx 20 columns, each containing either 0 or 1. It's the ...
1
vote
1answer
44 views

Why is PCA often used before t-sne for problems when the goal is only to reduce dimensionality?

Ex: Matlab's t-sne tutorials frequently use PCA https://www.mathworks.com/help/stats/tsne-settings.html " Process Data Using t-SNE Obtain two-dimensional analogs of the data clusters using t-SNE. ...
0
votes
0answers
66 views

Oddly shaped t-SNE visualizations with Word2Vec — CBOW vs Skipgram

I've been using t-SNE to graph some Gensim Word2Vec models trained on a relatively small corpus (10 epochs). For some reason, when graphed using t-SNE, the CBOW model has a cubic-like shape, whereas ...
1
vote
0answers
50 views

Approximating t-SNE embeddings for out-of-sample data

I have a large amount of data which has been reduced to two dimensions using t-SNE. Additional data points keep arriving, which I would like two-dimensional embeddings for, but this cannot be achieved ...
1
vote
0answers
42 views

Is there a representation of the separating hyperplane in t-sne?

I have used t-sne to visualize a set of images which I have used for training a binary classifier. Let us assume that the binary classifier is trained to detect cat(1) vs. no-cat(0). I have used the ...
2
votes
1answer
68 views

How to reduce position changes after dimensionality reduction?

Disclaimer: I'm a machine learning beginner. I'm working on visualizing high dimensional data (text as tdidf vectors) into the 2D-space. My goal is to label/modify those data points and recomputing ...
0
votes
0answers
18 views

How does t-sne scale work? Is it possible to compare scales of two t-sne plots?

I have used t-sne for two convolutional layers of binary classifier (VGG16 like CNN). I want to compare them, but I am not sure if that is okay as the scales of these two outputs from t-SNE plots can ...
2
votes
2answers
161 views

Grouping already clustered data (with a pre-defined x and y)

I have an already clustered data set (I wanna keep my x and y), where there's clearly a small group of elements in the middle that don't follow the expected patterns. I can select them manually, but ...
2
votes
2answers
396 views

Using t-SNE to track progress of a word vector embedding model. Pitfalls?

I've been training a word2vec/doc2vec model on a large amount of text. I recently stumbled across the t-SNE package, and am finding it wonderful at finding hidden structure in high-dimensional data. ...
1
vote
0answers
114 views

tsne plot not showing all the labels?

My data has 13 labels in total. But in tsne plot, the figure just displays 10 labels with the other 2 labels as "Other" in the same color. How can I set the plot to show all of the labels? Thanks.
2
votes
0answers
135 views

t-SNE on extremely high-dimensional spaces

I successfully applied t-SNE to the number handwriting dataset. n=3823 data points (i.e. handwritten numbers) in an D=64 dimensional space (i.e. 8x8 pixels). Worked great. Now I would like to cluster ...
0
votes
1answer
233 views

Good t-SNE or bad t-SNE?

I have used tsne to visualise a large dataset and it has produced the following graph. I need help interpreting it, as I have never seen a tsne graph like this before! I am aware that not much ...
1
vote
1answer
499 views

Reconstructing original data points from t-SNE output

I have been trying to understand t-SNE for a while now and I have this very basic question on the comparison of PCA and t-SNE, on which I would really appreciate some help. In case of PCA suppose the ...
5
votes
2answers
636 views

What does the long curve-shape t-SNE mean?

I use 1-D CNN input 1*512 size time series data which randomly fragment segment, the output will classify input into 10 classes. After training the CNN, I apply t-SNE to the prediction which I fed in ...
1
vote
1answer
156 views

Does it make sense to visualize data with a linear relationship using tSNE?

I have used tSNE several times to visualize high dimensional data for cluster analysis, and it has always worked quite well when data falls into clusters. However, does it make any sense to use tSNE ...
0
votes
1answer
625 views

How to recreate T-SNE dimensions deterministically?

So I have a set of 3000 features from which I would like to generate clusters. I passed my features through the T-SNE algorithm to reduce dimensionality to 2 features, and clusters are really visible ...
1
vote
1answer
222 views

Is there t-SNE in WEKA?

I want to use t-SNE in WEKA just for visualization purposes. I tried to look at the package manager but could not find any implementation of it. Is there anything that I can do to achieve it?
4
votes
2answers
4k views

How to create interactive plot of thousands of images as output of t-SNE?

I have many images that I want to plot as a result of running t-SNE and I want to be able to interactively explore them. matplotlib does not allow enough interactivity to explore, and plotly is too ...
1
vote
1answer
339 views

Converting similarity matrix before inputting to t-sne

I have a cosine similarity matrix where I want to adjust it to inputto t-sne. I fond the following explanation in a FAQ. As mentioned there I have made the diagonals to zero. what does it mean by <...
1
vote
1answer
464 views

Cluster doc2vec using Affinity Propagation

I want to cluster my document vectors (doc2vec) using affinity propagation. However, I am just confused if I should use cosine similarity or cosine distance to cluster my document vectors. Currently, ...
1
vote
1answer
1k views

Simple way to visualise word2vec vector space

I want to visualize my word2vec vector space (with zoom in and zoom out). I found a really interesting GitHub project named word2vec explorer. However, it does not seem to work in windows and mac. ...
2
votes
0answers
1k views

t-SNE plotting DBSCAN clustering results very scattered issue

We are trying a DBSCAN clustering model on our 30,000 samples with 15 features each. We tuned the epsilon parameter small enough to make sure the radius of the clustering circle is small while it does ...
3
votes
1answer
123 views

Dimensionality reduction with prior knowledge of colinearity between features

Let's say that I have sparse feature vectors and I'd like to use dimensionality reduction in order to visualize them more easily. Dimensionality reduction techniques like PCA will estimate ...
8
votes
2answers
147 views

Ways to reconstruct shuffled pixels of a video file?

Suppose that you have a video file which pixel order has been shuffled once. That is, a random order have been defined once and applied to all frames. Does it exist some known approach for ...
4
votes
3answers
3k views

tsne for prediction

I have a traditional prediction setting, with a training data set train and a test data set test. I do not know the outcome <...
10
votes
1answer
3k views

Does nearest neighbour make any sense with t-SNE?

Answers on here have stated that the dimensions in t-SNE are meaningless, and that the distances between points are not a measure of similarity. However, can we say anything about a point based on it'...
0
votes
1answer
241 views

How do I calculate a similarity matrix with a Student-t kernel?

As the title says, how do I calculate a similarity matrix with an un-normalized Student-t kernel? I'm attempting to calculate Kullback-Leibler divergence for different t-SNE runs, but need a Q-matrix ...
9
votes
1answer
1k views

t-SNE: Why equal data values are visually not close?

I have 200 data points that have the same values on all features. After t-SNE dimension reduction they doesn't look so equal anymore, just like this: Why aren't they on the same point in the ...
1
vote
1answer
885 views

Retain similarity distances when using an autoencoder for dimensionality reduction

I am trying to reduce the dimensionality of topic vectors (300, 1) to a two dimensional space. This has been done with various methods (e.g. t-SNE and autoencoders). A published example of reducing ...
2
votes
1answer
485 views

What are 2D dimensionality reduction algorithms good for?

It seems to me that t-SNE and other dimensionality reduction algorithms which reduce the dimensionality to two dimensions are mainly used to get an impression of the dataset. If done well, they look ...
18
votes
1answer
1k views

Are t-sne dimensions meaningful?

Are there any meanings for the dimensions of a t-sne embedding? Like with PCA we have this sense of linearly transformed variance maximizations but for t-sne is there intuition besides just the space ...
4
votes
1answer
810 views

Given a t-SNE plot, how can I infer the “most correct” labels? How does one understand its structure?

Let's say I begin with an exceptionally large dataframe (e.g. imported/munged from tsv files). Several of these columns are categorical labels. (As a more concrete example, let's imagine a group of ...
7
votes
2answers
8k views

Clustering high dimensional data

TL;DR: Given a big image dataset (around 36 GiB of raw pixels) of unlabeled data, how can I cluster the images (based on the pixel values) without knowing the number of clusters ...
2
votes
1answer
558 views

Organize TSNE data into grid

I have some data reduced by TSNE into a 2D representation, which shows clear spatial features. However, I'd like to format this into a grid – not just snapping data to the nearest grid square but ...
3
votes
2answers
246 views

How to equalize the pairwise affinity perplexities when implementing t-SNE?

I'm trying to implement the t-SNE algorithm: I found that to compute the pairwise affinities, I have to follow this: My problem is computing $\sigma_i$. In the Wikipedia I found: The bandwidth of ...
7
votes
4answers
372 views

Is t-SNE just for visualization?

I have used the t-SNE algorithm to visualize my high dimensional data. However, I was wondering if this is a practical method for inference?
6
votes
2answers
5k views

How to calculate KL-divergence between matrices

Given there are two matrices of dimensionality 100x2 with absolute values ranging from -50 to +50. Is it possible to determine the kl-divergence by applying the entropy algorithm from scipy.stats to ...
4
votes
1answer
221 views

Can I apply Clustering algorithms to the result of Manifold Visualization Methods?

Some methods related to manifold-learning are commonly stated as good-for-visualization, such as T-SNE and self-organizing-maps (SOM). I understand that when referring specifically to "visualization" ...
14
votes
1answer
2k views

Can closer points be considered more similar in T-SNE visualization?

I understand from Hinton's paper that T-SNE does a good job in keeping local similarities and a decent job in preserving global structure (clusterization). However I'm not clear if points appearing ...