5
$\begingroup$

I am trying to perform k-means clustering on multiple columns. My data set is composed of 4 numerical columns and 1 categorical column. I already researched previous questions but the answers are not satisfactory.

I know how to perform the algorithm on two columns, but I'm finding it quite difficult to apply the same algorithm on 4 numerical columns.

I am not really interested in visualizing the data for now, but in having the clusters displayed in the table.The picture shows that the first row belongs to cluster number 2, and so on. That is exactly what I need to achieve, but using 4 numerical columns, therefore each row must belong to a certain cluster.

Do you have any idea on how to achieve this? Any idea would be of great help. Thanks in advance! :enter image description here

$\endgroup$
  • $\begingroup$ Note that the age attribute is effectively ignored. You get the same result using only income. Because the data is not appropriately prepared for this analysis. $\endgroup$ – Has QUIT--Anony-Mousse Apr 5 '19 at 21:16
10
$\begingroup$

There is no difference in methodology between 2 and 4 columns. If you have issues then they are probably due to the contents of your columns. K-Means wants numerical columns, with no null/infinite values and avoid categorical data. Here I do it with 4 numerical features:

import pandas as pd
from sklearn.datasets.samples_generator import make_blobs
from sklearn.cluster import KMeans

X, _ = make_blobs(n_samples=10, centers=3, n_features=4)

df = pd.DataFrame(X, columns=['Feat_1', 'Feat_2', 'Feat_3', 'Feat_4'])

kmeans = KMeans(n_clusters=3)

y = kmeans.fit_predict(df[['Feat_1', 'Feat_2', 'Feat_3', 'Feat_4']])

df['Cluster'] = y

print(df.head())

Which outputs:

     Feat_1    Feat_2    Feat_3    Feat_4  Cluster
0  0.005875  4.387241 -1.093308  8.213623        2
1  8.763603 -2.769244  4.581705  1.355389        1
2 -0.296613  4.120262 -1.635583  7.533157        2
3 -1.576720  4.957406  2.919704  0.155499        0
4  2.470349  4.098629  2.368335  0.043568        0
| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for your help Simon! $\endgroup$ – Lina Apr 5 '19 at 14:06
  • $\begingroup$ Thanks @simon-larsson . How can we plot the output with many features ? :) $\endgroup$ – aniltilanthe Feb 6 at 16:02
  • $\begingroup$ Use dimensionality reducing techniques such as PCA or t-SNE to describe your features in two dimensions. Here is an example of t-SNE being used to plot MNIST in two dimensions. $\endgroup$ – Simon Larsson Feb 6 at 20:49
6
$\begingroup$

Let's take as an example the Breast Cancer Dataset from the UCI Machine Learning.

This is how it looks

>> _data.head(5)

   Age        BMI  Glucose  Insulin      HOMA   Leptin  Adiponectin  Resistin  \
0   48  23.500000       70    2.707  0.467409   8.8071     9.702400   7.99585   
1   83  20.690495       92    3.115  0.706897   8.8438     5.429285   4.06405   
2   82  23.124670       91    4.498  1.009651  17.9393    22.432040   9.27715   
3   68  21.367521       77    3.226  0.612725   9.8827     7.169560  12.76600   
4   86  21.111111       92    3.549  0.805386   6.6994     4.819240  10.57635   

     MCP.1  Classification  
0  417.114               1  
1  468.786               1  
2  554.697               1  
3  928.220               1  
4  773.920               1 

As you can see, all the columns are numerical. Let's see now, how we can cluster the dataset with K-Means. We don't need the last column which is the Label.

### Get all the features columns except the class
features = list(_data.columns)[:-2]

### Get the features data
data = _data[features]

Now, perform the actual Clustering, simple as that.

clustering_kmeans = KMeans(n_clusters=2, precompute_distances="auto", n_jobs=-1)
data['clusters'] = clustering_kmeans.fit_predict(data)

There is no difference at all with 2 or more features. I just pass the Dataframe with all my numeric columns.

   Age        BMI  Glucose  Insulin      HOMA   Leptin  Adiponectin  Resistin  \
0   48  23.500000       70    2.707  0.467409   8.8071     9.702400   7.99585   
1   83  20.690495       92    3.115  0.706897   8.8438     5.429285   4.06405   
2   82  23.124670       91    4.498  1.009651  17.9393    22.432040   9.27715   
3   68  21.367521       77    3.226  0.612725   9.8827     7.169560  12.76600   
4   86  21.111111       92    3.549  0.805386   6.6994     4.819240  10.57635   

   cluster  
0        0  
1        0  
2        0  
3        0  
4        0 

How you can visualize the clustering now? Well, you cannot do it directly if you have more than 3 columns. However, you can apply a Principal Component Analysis to reduce the space in 2 columns and visualize this instead.

### Run PCA on the data and reduce the dimensions in pca_num_components dimensions
reduced_data = PCA(n_components=pca_num_components).fit_transform(data)
results = pd.DataFrame(reduced_data,columns=['pca1','pca2'])

sns.scatterplot(x="pca1", y="pca2", hue=data['clusters'], data=results)
plt.title('K-means Clustering with 2 dimensions')
plt.show()

Here are the imports I used

import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
from sklearn.cluster import KMeans

And this is the visualization

K-Means after PCA

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Thank you Tasos for your response, very helpful! As I'm still quite new to this, I was wondering if it's normal for Pandas to display just the first 9 columns when computing the clustering? $\endgroup$ – Lina Apr 6 '19 at 13:34
  • 1
    $\begingroup$ The dataset I used has just 9 columns. So, that's all of it. $\endgroup$ – Tasos Apr 6 '19 at 13:44
  • $\begingroup$ Pardon me, I meant rows, not columns. $\endgroup$ – Lina Apr 6 '19 at 13:56
  • $\begingroup$ I am not sure what you mean. When you cluster, the whole dataset is used. Not just the first X rows. Maybe you can elaborate more? $\endgroup$ – Tasos Apr 6 '19 at 14:04
  • $\begingroup$ Sure. In my data set I have 4 columns composed of 64 rows each. Once I clustered, I expect a result of 64 rows, instead of just 9. I'm wondering if that is how it works or maybe I need to add a couple of lines of code to display the whole data frame? $\endgroup$ – Lina Apr 6 '19 at 14:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.