One-hot-encoded variables dominating clustering

Question

I am performing some unsupervised clustering with k-means on some transaction data that contains the following information:

Customer	units purchased in category_1	units purchased in category_1	time of day of purchase
a	1	2	Morning
b	3	2	Evening

To deal with the categorical variable time of day of purchase I have used one-hot-encoding. I have also used MinMax scaling on the data (so my units purchased in category x variable maxes out at 1):

Customer	units purchased in category_1	units purchased in category_1	Morning	Afternoon	Evening
a	0.1	0.2	1	0	0
b	0.3	0.2	0	0	1

What I am finding is that as the Morning, Afternoon, Evening variables are binary (0,1) they dominate the k-means clustering vs the units purchased, which are much smaller than 1 most of the time due to scaling. I end up with three clusters determined almost soley by the time of day.

How can I address this? Is there a better encoding method I could use or is k-mean just not suitable in this case?

Answer 1

Try to use polinomial coding:

import category_encoders as ce

encoder = ce.PolynomialEncoder(cols=["col_name"])
encoder.fit_transform(obj_df, verbose=1).head()

Doc: https://contrib.scikit-learn.org/category_encoders/polynomial.html

Answer 2

Time can be encoded in a variety of ways. You could choose a continuous encoding to be amenable for k-means. An example is minutes from 0:00 in 24-hour time notation which captures the concepts of "Morning", "Afternoon", and "Evening".

Answer 3

Kmode addresses your problem

Kmeans algorithms are best suited for clustering large datasets, however it limits its usage to numerical value

Kmodes on other hand, extends kmeans paradigm to categorical domains and is also able to cluster mixed data as mentioned in this paper, A Fast Clustering Algorithm to Cluster Very Large Categorical Data Sets in Data Mining

k-modes is used for clustering categorical variables. It defines clusters based on the number of matching categories between data points. (This is in contrast to the more well-known k-means algorithm, which clusters numerical data based on Euclidean distance.) The k-prototypes algorithm combines k-modes and k-means and is able to cluster mixed numerical / categorical data.

The kmodes algorithm have three major modifications to the k-means algorithm,

i.e.,

• using different dissimilarity measures,
• replacing k means with k modes, and
• using a frequency based method to update modes.

USAGE :

import numpy as np
from kmodes.kmodes import KModes

# random categorical data
data = np.random.choice(20, (100, 10))

km = KModes(n_clusters=4, init='Huang', n_init=5, verbose=1)

clusters = km.fit_predict(data)

# Print the cluster centroids
print(km.cluster_centroids_)

References

• Official code documentation

• Code implementation examples in Python

1. Example 1: applied directly on categorical

x = ["Dog", "Blue", "Female", "Sad"]
y = ["Cat", "Yellow", "Male", "Happy"]
z = ["Sheep", "Yellow", "Male", "Happy"]
a = ["Sheep", "Yellow", "Female", "Happy"]

df2 = pd.DataFrame([x,y,z,a], columns= ["Pet", "Sky", "Gender", "Feeling"])

km_2 = KModes(n_clusters=2, init="Huang")
km_2.fit_predict(df2)

km_2.cluster_centroids_

2. Example for numerical

x = [0,1,0]
y = [0,1,1]
z = [1,0,1]
a = [1,0,1]
b = [1,0,0]

df = pd.DataFrame([x,y,z, a, b], columns= ["Pet", "Sky", "Gender"])

km = KModes(n_clusters=2, init='Huang')
result = km.fit_predict(df)

km.cluster_centroids_

Out[14]:
array([[1, 0, 1],
       [0, 1, 0]])

In [15]:
km.labels_

3. Example 3 with categorical and numerical data

iris_df = pd.read_csv("../input_data/iris.csv")
iris_df.head()

from kmodes.kprototypes import KPrototypes
kP = KPrototypes(n_clusters=3, init='Huang', n_init=1, verbose=True)

kP.fit_predict(iris_df, categorical=[5])

      
kP.cluster_centroids_

Out[28]:
[array([[125.5  ,   6.588,   2.974,   5.552,   2.026],
        [ 25.5  ,   5.006,   3.428,   1.462,   0.246],
        [ 75.5  ,   5.936,   2.77 ,   4.26 ,   1.326]]), array([['virginica'],
        ['setosa'],
        ['versicolor']], dtype='<U10')]


iris_df["cluster_id"] = kP.labels_

# testing to confirm
iris_df[iris_df.Species == 'versicolor']