Hot answers tagged

17

The dimensions of the low dimensional space have no meaning. Note that the t-SNE loss function is solely based on the distances between points ($y_i$ and $y_j$) and probability distributions over those distances ($p_{ij}$ and $q_{ij}$): $$ \frac{\delta C}{\delta y_i}=4 \sum_j(p_{ij} - q_{ij})(y_i-y_j)(1+||y_i -y_j||^2)^{-1} $$ Thus there is no projection ...


11

I don't think any of the clustering techniques "just" work at such scale. The most scalable supposedly is k-means (just do not use Spark/Mahout, they are really bad) and DBSCAN (there are some good distributed versions available). But you will be facing many other challenges besides scale because clustering is difficult. It's not as if it's just enough to ...


8

No, it is not necessary that this is the case, however, this is, in a convoluted way, the goal of T-SNE. Before getting into the meat of the answer, let's take a look at some basic definitions, both mathematically and intuitively. Nearest Neighbors: Consider a metric space $\mathbb{R}^d$ and a set of vectors $X_1, ..., X_n \in \mathbb{R}^d$, given a new ...


5

I would present t-SNE as a smart probabilistic adaptation of the Locally-linear embedding. In both cases, we attempt to project points from a high dimensional space to a small one. This projection is done by optimizing the conservation of local distances (directly with LLE, preproducing a probabilistic distribution and optimizing the KL-divergence with t-SNE)...


4

There seem to be a few options, but I found rasterfairy which is very easy to install and use. Has the added bonus of being able to fit to a rectangular grid, but also circular and other arbitrary shapes. A very nice IronPython notebook example: https://github.com/Quasimondo/RasterFairy/blob/master/examples/Raster%20Fairy%20Demo%201.ipynb And some example ...


4

With t-SNE none of the input parameters are weighted more than any other parameter so the differences you want to see like students forming islands by grade level will not happen because there is so much other data present to pull those students/data points in different directions. I highly encourage you to have a specific question in mind and tailor your ...


4

A general solution to this does not exist, even if we add some assumptions about the distribution of e.g. colours and shapes in the images or temporal coupling such as consecutive frames being similar. Problem Let $F_1,\dots,F_i$ be the $n$ original frames, each with $m$ pixels. Let $P$ be the permutation that is applied to the pixels of each frame before ...


4

Datashader is a Python visualization library designed to handle large datasets. A tutorial to plot t-SNE with datashader can be found here.


4

It is certainly possible to create interactive plots of many thousands of images, as Google has done in their interactive art t-SNE Map. However, as far as I have found, there is not any canned way to do this. https://github.com/YaleDHLab/pix-plot is an open source application that also does an impressive job at plotting thousands of images, and if you are ...


3

This is a fascinating combinatorial problem. I would featuring each pixel using its full temporal trajectory, then embed them in a grid using the k nearest neighbors. The real goal is to maximize the likelihood of the video being a sequence of natural (real life) images, which you can test with a classifier, but you might be able to get away with just a ...


3

You can easily visualize word2vec vectors using TensorBoard, which is very easy to understand. I have made a video about using TensorBoard.


3

You're correct that the same values in T-SNE can be distributed across different points, the reason this happens is clear if you take a look at the algorithm that T-SNE runs across. To address your first concern about the points actually not being the same after the algorithm has been applied to the dataset. I will leave you with an exercise to verify it ...


3

You can do anything you want in the low dimensional space, and can try to validate as well. By clustering the above, you are in effect assigning features/tags to your data points in higher dimensions. Remember, tSNE tries to preserve distances, so that points in high dimensions will remain close to each other in low dimensions. With that in mind, don't ...


3

No. t-Distributed Stochastic Neighbor Embedding (t-SNE) and Principal Component Analysis (PCA) are dimension reduction techniques, aka fewer columns of a tidy dataframe. Clustering will reduce the number of observations, aka fewer rows of a tidy dataframe. In particular, you might be looking for hierarchical clustering.


2

It's a dimensionality reduction algorithm. Inference is the problem of determining the parameters, or labels, that best fit the model for a given input once the model parameters have been learned, or estimated.


2

I think you can. Just normalize both of the vectors to be sure they are distributions. Then you can apply the kl divergence . Note the following: - you need to use a very small value when calculating the kl-d to avoid division by zero. In other words , replace any zero value with ver small value - kl-d is not a metric . Kl(AB) does not equal KL(BA) . If you ...


2

It simply means that you should set the bandwidths through binary search. The way it works is that you start with a preset target perplexity (Mark's link suggests values from 5 to 50 as reasonable values), and bounds for the bandwidth. If the target perplexity is inside the interval defined by the boundary perplexities, you iteratively halve the search space ...


2

No it is not possible to preserve relative distances when reducing dimensions for arbitrary data. This is not due to a property of auto-encoders compared to e.g. PCA or T-SNE. It is due to geometry. You can see this relatively easily by considering a reduction of dimensions from 3 to 2, and examining a tetrahedron where all four corner points are 1 unit ...


2

There are many questions there, I'll try to address each in turn. How is t-SNE* better than just taking a random (probably stratified) sample of the data? If your goal is to provide a visual overview of the data then clearly a stratified sample is not going to do that -- the samples each still live in high dimensional space, so are no easier to visualise, ...


2

Your assumption is right, the results are in general misleading. Suppose your (linearly correlated) data have missing points in some range: Than for t-SNE the two subset of data will be two different clusters, even if they lie on the same linear distribution: But, if you are actually interested in the fact that those two structures are separated, then t-...


2

Do not use tSNE visualizations for clustering. The results are misleading. See this great answer: https://stats.stackexchange.com/a/264647/7828 Apart from that, you just need to fix the initial positions. For examples by fixing the random generator seed. But the fact thar it does not work every time should already warn you that it is not too reliable...


2

T-SNE is extremely useful for visualizing high-dimensional data in lower-dimensional space. However, t-SNE can have several gotchas, including comparing cluster sizes. The t-sne algorithm tries to even out cluster sizes by expanding dense clusters and contracting sparse clusters. Thus, it is not straightforward to directly compare clusters across different ...


2

If you want to reduce the number of classes you are predicting over, then you could manually map them to a simpler set (i.e. map poodle, greyhound to dog ) OR if you don't have the domain knowledge you can cluster your data and predict the cluster instead of their original labels. You could use PCA or t-SNE to reduce the number of dimensions before ...


1

There is no closed form. It is a local embedding, and you really cannot expect to find a good inverse mapping. It's doable, but not very good. Use gradient descent in the input domain. Just like tSNE but with the t-distribution and the Gaussian reversed. However, the original Gaussian has a sigma parameter that you also need to find. So you need to also ...


1

It would make sense that the time-series data sticks together - and so forms these lines you are seeing. In normal time-series analysis where the variables are assumed to be random (e.g. modelled on Brownian motion), the best prediction for tomorrow is just the same as today. t-SNE finds the closest points withing your feature-space and embedding them into a ...


1

The lines therefore are the time series. t-sne is a very locally sensitive algorithm so every data point is very likely to show up nearest (in 2-D) to its nearest (N-D) neighbour. In MNIST there is no sequence, so it doesn't show up as a line in 2-D. Every digit is written in a set of similar ways that cluster together. In your time-series data, if each ...


1

Sadly no, there is not a T-SNE implementation for WEKA. If you can install python packages in your environment, then you can use the wekaPython package (in WEKA's package manager) to run scikit-learn's T-SNE implementation on data you have loaded into WEKA. Use this code in the 'CPython Scripting' panel (which appears after successfully installing ...


1

One option is to use Multidimensional scaling (MDS) for dimensionality reduction. MDS can create a visualization of the relative positions for data based on the distance between the data points. In your example, the data points that have no distance between them in several dimensions will be projected close to each other.


1

You need to scale the values by some constant factor so the sum of every entry in the matrix results in 1.0. You can achieve this by using mat /= mat.sum(), where mat is your matrix.


1

Both of them convert the distances back to similarities, albeit using different methods. They will, if I recall correctly, also square the distances. This may be problematic with the most common variant of cosine distance, which already is a squared distance. So it may be a good idea to modify the methods to be able to directly work with the similarities. ...


Only top voted, non community-wiki answers of a minimum length are eligible