7
$\begingroup$

I use 1-D CNN input 1*512 size time series data which randomly fragment segment, the output will classify input into 10 classes. After training the CNN, I apply t-SNE to the prediction which I fed in testing data. In general, the output shape of the tsne result is spherical(for example,applied on MNIST dataset). But now I apply t-SNE on my own dataset. No matter how I adjust perplexity early, learning rate or maximum number of iterations. It will give me the result of long-shaped output, just like the pic below. Does the long-shaped t-SNE mean anything? Thanks everyone beforehand.

enter image description here

Edit: Explaining about dataset contents. p.s. before feeding into CNN for training, I randomly split it into training/testing dataset. enter image description here

$\endgroup$
  • $\begingroup$ Can you edit your question to explain the data you have? Without knowing your data-set it will be hard to say anything for sure. $\endgroup$ – f.g. Jul 8 '18 at 11:18
  • $\begingroup$ No problem @f.g. this dataset is called CWRU bearing dataset. It contains different types of bearing fault time series data. I separate it to 10 different types. The detail I will edit supplement and explain above. Thank you~! $\endgroup$ – Chiao Wei Yeh Jul 8 '18 at 12:14
2
$\begingroup$

It would make sense that the time-series data sticks together - and so forms these lines you are seeing. In normal time-series analysis where the variables are assumed to be random (e.g. modelled on Brownian motion), the best prediction for tomorrow is just the same as today. t-SNE finds the closest points withing your feature-space and embedding them into a 2D space. It is quite impressive that it picks it out and ends up with your plot!

While you often get circular looking plots, it is not true that you always get circles/spheres. t-SNE is maximising the distance between clusters and at the same to minimising the distance between points within a single cluster... for the sake of efficiency, circles arise very often. You can observe this in nature: the shape of planets, of bubbles in water... circles are efficient!

t-SNE does not allow you to directly interpret the distance between clusters back to the input units (e.g. a line twice as high as another doesn't mean the values are twice as big). It would perhaps be interesting to plot the individual time-series lines themselves there (within a single cluster) next to the input time-series data of the same feature, then look for any correspondance.

For more understanding, I would recommend reading this great walkthrough/visualisation article covering t-SNE. There are a few examples that show non-circular results:

almost linear clusteres

Wattenberg, et al., "How to Use t-SNE Effectively"

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for ur reply! but what is " It would perhaps be interesting to plot the individual time-series lines themselves there (within a single cluster) next to the input time-series data of the same feature, then look for any correspondance "?? Can you slightly explain it? $\endgroup$ – Chiao Wei Yeh Jul 13 '18 at 22:54
1
$\begingroup$

The lines therefore are the time series. t-sne is a very locally sensitive algorithm so every data point is very likely to show up nearest (in 2-D) to its nearest (N-D) neighbour.

In MNIST there is no sequence, so it doesn't show up as a line in 2-D. Every digit is written in a set of similar ways that cluster together. In your time-series data, if each time point is most similar to its next time point, what we expect is for them to be in some sequence - because you'd want point t=n to be closest to points t=n-1 and t=n+1 (if they are in fact the most similar).

| improve this answer | |
$\endgroup$
-1
$\begingroup$

This is also likely to be due to the fact that you're using features from a softmax (or other probability) layer. Make sure you are using a layer of activations and not probabilities. For example, in Keras, if you extract features from the last layer, it extracts the softmax probability vector. Probability vectors are very similar to each other, especially if they belong to the same class, since this is what softmax tries to do. I have seen similar shapes using time-series data when extracting probabilities and then clouds of data (instead of curves) when extracting previous layer activations.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.