2
$\begingroup$

I'm using TSNE to visualize my clusters but the output seems a bit strange. There are supposed to be 3 clusters but instead, there are 4 lines. Is there something wrong with how I'm visualizing them or is it the kmeans method itself?

import pandas as pd
import numpy as np
import ast
from sklearn import metrics
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt
from sklearn.manifold import TSNE
from sklearn.decomposition import TruncatedSVD

colNames = ['unixTime', 'sampleAmount','Time','samplingRate', 'Data']

data = pd.read_csv("project_fan.csv",  sep = ';', error_bad_lines = False, names = colNames) 

# changing data into list
data['Data'] = data.Data.transform(ast.literal_eval)

# Selecting the average value from the list and replacing the list with it
data['Data'] = data.Data.apply(np.mean)

kmeanModel = KMeans(n_clusters = 3)
kmeanModel.fit(data)

y = kmeanModel.labels_


X_train, X_test, y_train, y_test = train_test_split(data, y, test_size = 0.2, random_state = 1)

k = 3
tfs_reduced = TruncatedSVD(n_components=k, random_state=0).fit_transform(data)
tfs_embedded = TSNE(n_components=2, perplexity=40, verbose=2).fit_transform(tfs_reduced)
fig = plt.figure(figsize = (10, 10))
ax = plt.axes()
plt.scatter(tfs_embedded[:, 0], tfs_embedded[:, 1], marker = "x", c = km.labels_)
plt.show()

Sample Dataset:

       unixTime  sampleAmount  Time  samplingRate         Data
0  1.556891e+09         16384   340  48188.235294  1620.242170
1  1.556891e+09         16384   341  48046.920821  1620.237716
2  1.556891e+09         16384   340  48188.235294  1620.236340
3  1.556891e+09         16384   340  48188.235294  1620.229289
4  1.556891e+09         16384   340  48188.235294  1620.227541

Output: enter image description here

$\endgroup$
  • $\begingroup$ I am not sure we will be able to help you without the data. How did you find that there are 3 clusters ? Can you plot the tnse results but with the kmeans colors ? $\endgroup$ – lcrmorin Jan 19 at 13:20
  • $\begingroup$ @lcrmorin here's my data: drive.google.com/file/d/1APIG7C5d-zWPfe1bZa2azDmfQIkDOyVu/… $\endgroup$ – x89 Jan 19 at 13:55
  • $\begingroup$ @lcrmorin I used the elbow method to find an optimal number of clusters i.e 3 $\endgroup$ – x89 Jan 19 at 13:55
1
$\begingroup$

As far as i see, you have your temporal columns involved. They are sequential variables and create such curves in visualization.

Try to exclude them from your data and visualize again. If you still the curves, update the post with new plot and we discuss further.

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.