K-means algorithm for multiple variables

Question

I am a new to ML and current in reading about K-Means algorithm and trying it out with ORANGE tool. After going through several examples on YouTube and various other places, I am slightly confused on how Silhouette calculation works for finding the K-value.

Most of the examples are showing that Silhouette calculation(s-score) is showing the K value based on the two dimension data they have. When I use it ORANGE tool, I first check the k value that has least s-score and using that as K value and then plotting the scatter plot which is very straightforward to understand.

However, when my data has multiple columns(like age, location, interests etc), how the silhoutte calculation works? based on which column? Can someone help me understand this?

Answer 1

Silhouette (and similar algorithms like the Davies–Bouldin index or the Dunn index) base their computation on the distance between data-points (typically, they compare the distances within clusters with distances inbetween them in some way). This means, that the algorithms themself kann work with any distance function between data-points.

What you have seen in the examples was probably the euclidean distance (that's what you understand as distance, when you look at the example). This distance works also with higher dimensions and can combine an arbitrary number of numeric columns to compute the distance.

With categorical columns, it is a bit more tricky, but there are well-established ways to do so (e.g. via one-hot-encoding).

Summary

Distances are crucial for many data science task. For that reason,

• all major frameworks can off-the-shelf compute distances between different samples, often with little to no preprocessing required.
• In some cases, it might make sense to use a different distance (a list of distances from Orange) or even define a custom one. Such decisions should be driven by the understanding of the use case, the data and the goal that the systam shall achieve.