Categorical features preprocessing for clustering

Question

Can anyone tell suggest the best practice for clustering data with mixtured features (both with categorical and continuous). I am struggling with a problem; I realized that for all metrics algorithms it is necessary to scale continuous data before clustering, so I used sklearn.preprocessing.StandardScaler(). With my categorical features, I used onehotencoder transformation, but it's not clear if it's necessary also scale these onehotencoded vector's components regarding to the whole data set or just leave them as they are?

Answer 1

Using Gower distance for the clustering. You can find different article about this measure in this and this article.

Gower Distance is a distance measure that can be used to calculate distance between two entity whose attribute has a mixed of categorical and numerical values.

Gower distance is computed as the average of partial dissimilarities across individuals. Each partial dissimilarity (and thus Gower distance) ranges in [0 1]. $$d(i,j) = \frac{1}{p}\sum_{i=1}^{p}d_{ij}^{(f)}$$

Partial dissimilarities ($d_{ij}^{(f)}$) computation depend on the type of variable being evaluated. This implies that a particular standardization will be applied to each feature, and the distance between two individuals is the average of all feature-specific distances.

• For a numerical feature $f$, partial dissimilarity is the ratio between 1) absolute difference of observations $x_i$ and $x_j$ and 2) maximum range observed from all individuals: $d_{ij}^{(f)} = \frac{|x_i - x_j|}{|(\max_N(x) - \min_N(x))|}$ , $N$ being the number of individuals in the dataset.

• Partial dissimilarity computation for numerical features ($R_f$ = maximal range observed) For a qualitative feature $f$ partial dissimilarity equals 1 only if observations $y_i$ and $y_j$ have different value. Zero otherwise.

Answer 2

The best practice is to first understand your problem.

All the standardization, one-hot-encoding etc. are just hacks to squeeze non-continuous data into column types that can pass as continuous if you don't look at the data too closely.

Yes, at some point you can then run k-means. But will you still have any idea what it does? Does it solve the right task? What bias does it have from all the encoding and scaling? Won't the results become completely useless because of dimensionality, discreteness, encoding artifacts, bias, ... - and what did you actually intend in the first place - I guess you didn't want random partitions...