0
$\begingroup$

Suppose we have this kind of data and the preferred clusters (not in the case of the optimal number of clusters, but the shape of clusters) here: enter image description here

I achieved the exact shape of clusters using KMeans! But I wonder which metric I should use to find the optimal number of clusters! I used silhouette coefficient, But it results in 2 clusters for these kinds of scattered data! So I'm slightly suspicious if I used a suitable metric for this problem.

To better explain this case, I want to dig into some details. One may ask what the x ticks and y ticks are.

  • x ticks are the index of data! like [1, 2, ... length of data]
  • y ticks shows the distance of each data point from a specific one.

So now it's a little bit clear what the idea is. I want data points within a distance range of (0, 🔴) to be in the same cluster, and distance range of (🔴, 🟡) in the same cluster, and so on. But the problem is I don't trust the silhouette coefficient because it always shows me 2 clusters through all varieties of data that I have in this specific problem!

Additional note:
the shape of clusters should be like the example (this is intended), no matter how scattered the data is.

$\endgroup$

1 Answer 1

0
$\begingroup$

https://www.scikit-yb.org/en/latest/api/cluster/elbow.html#:~:text=If%20the%20line%20chart%20looks,fits%20best%20at%20that%20point. I would recommend approaching the problem using the elbow method with the metric set as distortion ( by default) to find the optimal number of clusters.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.