Questions tagged [tsne]

t-SNE (t-distributed stochastic neighbor embedding) is a technique for dimensionality reduction.

Filter by
Sorted by
Tagged with
1
vote
0answers
29 views

How to plot centroids and clusters resulting from a KMean model based on a text variable

I hope you can help as after several attempts, I'm no longer sure I can get a decent result. I have a text corpus made of several documents, like the one below (which has been simplified for the sake ...
0
votes
1answer
40 views

what information can we obtain from t-SNE?

I see that t-SNE can help us reduce dimensions and visualize the data. But what information are we gaining from this visualization? As we know that the new axis don't have a meaning in our context. ...
3
votes
0answers
68 views

What Clustering Method Should I Use?

My data is a group of 10 thousand points (each having an node location (x,y)) that are spread across a plane. They are also chromatically-colored based on their weight. I need to finalize a bayesian ...
0
votes
0answers
27 views

(UMAP) Why does umap.transform() put new data in the periphery of old clusters?

UMAP is a dimensionality reduction method like t-SNE and PCA. Unlike t-SNE, UMAP can incorporate new data and do a forward transform on the data. While experimenting, I made an interesting observation ...
0
votes
2answers
50 views

can I use t-sne or PCA to reduce number of classes?

I wanted to know if I can use t-sne or PCA to reduce the number of classes depending on the similarity between them. For example, if I have 100 classes of 100 different animals and would like to put ...
2
votes
1answer
129 views

Using TSNE to Visualize Clusters in Python

I'm using TSNE to visualize my clusters but the output seems a bit strange. There are supposed to be 3 clusters but instead, there are 4 lines. Is there something wrong with how I'm visualizing them ...
1
vote
1answer
128 views

comparison of t-SNE and PCA and truncate SVD

How to compare the trucate SVD ,PCA, and T-SNE? What we can say about features if t-SNE and PCA and truncate SVD digaram is in this figure?
0
votes
2answers
35 views

If in t-SNE digaram of binary classification both classes follow the similar curve what does t-SNE diagram show?

If in t-SNE digaram of binary classification both classes follow the similar curve what does t-SNE diagram show for instance: Figure1 or Figure2
1
vote
2answers
589 views

python tsne.transform does not exist?

I am trying to transform two datasets: x_train and x_test using tsne. I assume the way to do this is to fit tsne to x_train, and then transform x_test and x_train. But, I am not able to transform any ...
2
votes
1answer
88 views

long shape line in tSNE plot

My question is similar to this post: What does the long curve-shape t-SNE mean? but the problem is that my data is not time series, but I also get these long shape line in tSNE plot . I don't know ...
0
votes
1answer
121 views

Meaning of curved line shape distribution in t-SNE plot

I have blood test results with 20 features from 170 patients, and trying to predict a categorical disease outcome. The input features are all continuous values except sex (0:male, 1:female) There are ...
0
votes
0answers
16 views

Suggestions on non-linear dimensionality reduction for small, one-hot encoded dataset

I wish to apply non-linear dimensionality reduction on a very small dataset (less than 100 observations). The dataset is very sparse, of approx 20 columns, each containing either 0 or 1. It's the ...
1
vote
1answer
81 views

Why is PCA often used before t-sne for problems when the goal is only to reduce dimensionality?

Ex: Matlab's t-sne tutorials frequently use PCA https://www.mathworks.com/help/stats/tsne-settings.html " Process Data Using t-SNE Obtain two-dimensional analogs of the data clusters using t-SNE. ...
1
vote
0answers
90 views

Approximating t-SNE embeddings for out-of-sample data

I have a large amount of data which has been reduced to two dimensions using t-SNE. Additional data points keep arriving, which I would like two-dimensional embeddings for, but this cannot be achieved ...
1
vote
0answers
48 views

Is there a representation of the separating hyperplane in t-sne?

I have used t-sne to visualize a set of images which I have used for training a binary classifier. Let us assume that the binary classifier is trained to detect cat(1) vs. no-cat(0). I have used the ...
3
votes
1answer
73 views

How to reduce position changes after dimensionality reduction?

Disclaimer: I'm a machine learning beginner. I'm working on visualizing high dimensional data (text as tdidf vectors) into the 2D-space. My goal is to label/modify those data points and recomputing ...
2
votes
2answers
176 views

Grouping already clustered data (with a pre-defined x and y)

I have an already clustered data set (I wanna keep my x and y), where there's clearly a small group of elements in the middle that don't follow the expected patterns. I can select them manually, but ...
2
votes
2answers
540 views

Using t-SNE to track progress of a word vector embedding model. Pitfalls?

I've been training a word2vec/doc2vec model on a large amount of text. I recently stumbled across the t-SNE package, and am finding it wonderful at finding hidden structure in high-dimensional data. ...
1
vote
0answers
125 views

tsne plot not showing all the labels?

My data has 13 labels in total. But in tsne plot, the figure just displays 10 labels with the other 2 labels as "Other" in the same color. How can I set the plot to show all of the labels? Thanks.
2
votes
0answers
139 views

t-SNE on extremely high-dimensional spaces

I successfully applied t-SNE to the number handwriting dataset. n=3823 data points (i.e. handwritten numbers) in an D=64 dimensional space (i.e. 8x8 pixels). Worked great. Now I would like to cluster ...
0
votes
1answer
303 views

Good t-SNE or bad t-SNE?

I have used tsne to visualise a large dataset and it has produced the following graph. I need help interpreting it, as I have never seen a tsne graph like this before! I am aware that not much ...
1
vote
1answer
690 views

Reconstructing original data points from t-SNE output

I have been trying to understand t-SNE for a while now and I have this very basic question on the comparison of PCA and t-SNE, on which I would really appreciate some help. In case of PCA suppose the ...
8
votes
3answers
982 views

What does the long curve-shape t-SNE mean?

I use 1-D CNN input 1*512 size time series data which randomly fragment segment, the output will classify input into 10 classes. After training the CNN, I apply t-SNE to the prediction which I fed in ...
1
vote
1answer
204 views

Does it make sense to visualize data with a linear relationship using tSNE?

I have used tSNE several times to visualize high dimensional data for cluster analysis, and it has always worked quite well when data falls into clusters. However, does it make any sense to use tSNE ...
0
votes
1answer
770 views

How to recreate T-SNE dimensions deterministically?

So I have a set of 3000 features from which I would like to generate clusters. I passed my features through the T-SNE algorithm to reduce dimensionality to 2 features, and clusters are really visible ...
1
vote
1answer
251 views

Is there t-SNE in WEKA?

I want to use t-SNE in WEKA just for visualization purposes. I tried to look at the package manager but could not find any implementation of it. Is there anything that I can do to achieve it?
4
votes
2answers
4k views

How to create interactive plot of thousands of images as output of t-SNE?

I have many images that I want to plot as a result of running t-SNE and I want to be able to interactively explore them. matplotlib does not allow enough interactivity to explore, and plotly is too ...
1
vote
1answer
384 views

Converting similarity matrix before inputting to t-sne

I have a cosine similarity matrix where I want to adjust it to inputto t-sne. I fond the following explanation in a FAQ. As mentioned there I have made the diagonals to zero. what does it mean by <...
1
vote
1answer
493 views

Cluster doc2vec using Affinity Propagation

I want to cluster my document vectors (doc2vec) using affinity propagation. However, I am just confused if I should use cosine similarity or cosine distance to cluster my document vectors. Currently, ...
1
vote
1answer
1k views

Simple way to visualise word2vec vector space

I want to visualize my word2vec vector space (with zoom in and zoom out). I found a really interesting GitHub project named word2vec explorer. However, it does not seem to work in windows and mac. ...
2
votes
0answers
1k views

t-SNE plotting DBSCAN clustering results very scattered issue

We are trying a DBSCAN clustering model on our 30,000 samples with 15 features each. We tuned the epsilon parameter small enough to make sure the radius of the clustering circle is small while it does ...
3
votes
1answer
129 views

Dimensionality reduction with prior knowledge of colinearity between features

Let's say that I have sparse feature vectors and I'd like to use dimensionality reduction in order to visualize them more easily. Dimensionality reduction techniques like PCA will estimate ...
9
votes
2answers
167 views

Ways to reconstruct shuffled pixels of a video file?

Suppose that you have a video file which pixel order has been shuffled once. That is, a random order have been defined once and applied to all frames. Does it exist some known approach for ...
4
votes
3answers
4k views

tsne for prediction

I have a traditional prediction setting, with a training data set train and a test data set test. I do not know the outcome <...
10
votes
1answer
3k views

Does nearest neighbour make any sense with t-SNE?

Answers on here have stated that the dimensions in t-SNE are meaningless, and that the distances between points are not a measure of similarity. However, can we say anything about a point based on it'...
0
votes
1answer
262 views

How do I calculate a similarity matrix with a Student-t kernel?

As the title says, how do I calculate a similarity matrix with an un-normalized Student-t kernel? I'm attempting to calculate Kullback-Leibler divergence for different t-SNE runs, but need a Q-matrix ...
9
votes
1answer
1k views

t-SNE: Why equal data values are visually not close?

I have 200 data points that have the same values on all features. After t-SNE dimension reduction they doesn't look so equal anymore, just like this: Why aren't they on the same point in the ...
1
vote
1answer
940 views

Retain similarity distances when using an autoencoder for dimensionality reduction

I am trying to reduce the dimensionality of topic vectors (300, 1) to a two dimensional space. This has been done with various methods (e.g. t-SNE and autoencoders). A published example of reducing ...
2
votes
1answer
516 views

What are 2D dimensionality reduction algorithms good for?

It seems to me that t-SNE and other dimensionality reduction algorithms which reduce the dimensionality to two dimensions are mainly used to get an impression of the dataset. If done well, they look ...
18
votes
1answer
1k views

Are t-sne dimensions meaningful?

Are there any meanings for the dimensions of a t-sne embedding? Like with PCA we have this sense of linearly transformed variance maximizations but for t-sne is there intuition besides just the space ...
4
votes
1answer
880 views

Given a t-SNE plot, how can I infer the “most correct” labels? How does one understand its structure?

Let's say I begin with an exceptionally large dataframe (e.g. imported/munged from tsv files). Several of these columns are categorical labels. (As a more concrete example, let's imagine a group of ...
9
votes
2answers
9k views

Clustering high dimensional data

TL;DR: Given a big image dataset (around 36 GiB of raw pixels) of unlabeled data, how can I cluster the images (based on the pixel values) without knowing the number of clusters ...
3
votes
1answer
598 views

Organize TSNE data into grid

I have some data reduced by TSNE into a 2D representation, which shows clear spatial features. However, I'd like to format this into a grid – not just snapping data to the nearest grid square but ...
3
votes
2answers
262 views

How to equalize the pairwise affinity perplexities when implementing t-SNE?

I'm trying to implement the t-SNE algorithm: I found that to compute the pairwise affinities, I have to follow this: My problem is computing $\sigma_i$. In the Wikipedia I found: The bandwidth of ...
7
votes
4answers
465 views

Is t-SNE just for visualization?

I have used the t-SNE algorithm to visualize my high dimensional data. However, I was wondering if this is a practical method for inference?
6
votes
2answers
6k views

How to calculate KL-divergence between matrices

Given there are two matrices of dimensionality 100x2 with absolute values ranging from -50 to +50. Is it possible to determine the kl-divergence by applying the entropy algorithm from scipy.stats to ...
4
votes
1answer
228 views

Can I apply Clustering algorithms to the result of Manifold Visualization Methods?

Some methods related to manifold-learning are commonly stated as good-for-visualization, such as T-SNE and self-organizing-maps (SOM). I understand that when referring specifically to "visualization" ...
14
votes
1answer
3k views

Can closer points be considered more similar in T-SNE visualization?

I understand from Hinton's paper that T-SNE does a good job in keeping local similarities and a decent job in preserving global structure (clusterization). However I'm not clear if points appearing ...