How to create new feature based on clustering result

Question

I'm still a beginner in machine learning and I want to know how to code this situation based on python and machine learning (clustering).

I have data like:

id      Column1    duration(seconde)    column3
1       aaa        20                   bbb
2       ccc        01                   ddd
3       eee        150                  fff
4       ggg        25                   hhh

I want to group my data according to the duration column value and create new column containing a category name based on duration cluster. I want to get this result:

id      Column1    duration(seconde)    column3      NewColCategorie
1       aaa        20                   bbb          Cat2 
2       ccc        01                   ddd          Cat1
3       eee        150                  fff          Cat3
4       ggg        25                   hhh          Cat2
5       iii        175                  jjj          Cat3

Answer 1

To do clustering you can use sklearn's KMeans Clustering function - sklearn.cluster.KMeans with n_clusters=3 and other parameters as default. This will give you 3 clusters. After you have trained your model you can use the .labels_ attribute of the trained model to classify every example. You can do this in the following way:

>>> from sklearn.cluster import KMeans
>>> import numpy as np
>>> X = np.array([[1, 2], [1, 4], [1, 0],
...               [10, 2], [10, 4], [10, 0]])
>>> kmeans = KMeans(n_clusters=2, random_state=0).fit(X)
>>> kmeans.labels_
array([1, 1, 1, 0, 0, 0], dtype=int32)

To create a new column based on category cluster you can simply add the kmeans.labels_ array as a column to your original dataframe:

>>> df['categories'] = kmeans.labels_

Answer 2

Here, is another way to use clustering for creating a new feature. We pass the input_data to fit_predict and store the result in new col_name.

Note: just remember that need to standardize data before performing clustering.

from sklearn.cluster import KMeans

kmeans = KMeans(n_clusters=4, max_iter=500, init="k-means++", tol=0.001)
X = np.array([[1, 2], [1, 4], [1, 0],
              [10, 2], [10, 4], [10, 0]])

X["cluster"] = kmeans.fit_predict(X)
X["cluster"] = X["cluster"].astype("category")
```