2
$\begingroup$

I know that for hierarchical clustering, it's the best practice to scale before so that you give the same weight to each variable. Otherwise, for the complete linkage, the variable with a wider range would be a dominant factor when determining which clusters to combine.

My question is, for the single linkage without scaling, is the variable with a wider range would be a dominant factor when determining which clusters to combine as the complete linkage? thanks in advance

$\endgroup$

1 Answer 1

2
$\begingroup$
Brief Summary

Yes, a wider-range-variable would dominate the single linkage clustering without scaling.

Explanation

The tendency of wider-range-variables to dominate the clustering does not only apply to hierachical clustering, but to many clustering methods.

The reason for this lies below the clustering: most (if not every) clustering algorithm is based on a distance metric. If not otherwise specified, the euclidean distance is typically uses. And this metric is dominated by the wide-range variables. Hence, the clustering algorithms that rely on such a metric, are as well dominated by the wider-range-variables.

Normalizing is the easiest way to handle this problem (if it is a problem). Using different metrics would be another way. E.g. the Mahalanobis distance does kind of a normalization by it self. Another approach would be a custom metric that uses some domain knowledge.

Example

Do demonstate this, I created a example dataset with

  • wide-range y-axis and small-range x-Axis (left column)
  • normalized features (right column)

And clustered it with

  • complete linkage (top row)
  • singale linkage (bottom row) enter image description here

As you can see, the non-normalized data (left column) is clustered nearly exclusivly by the y-value.

You also see that the complete linkage prefers compact clusters (top row, both columns), while the single linkage avoids bigger gaps (bottom row, right image)

$\endgroup$
14
  • $\begingroup$ Kudos to this answer and the linked website. $\endgroup$
    – Broele
    Commented Sep 11, 2023 at 15:58
  • $\begingroup$ Hi Broele. Just have a question. You said at the end that complete linkage prefers compact clusters (top row, both columns), while the single linkage avoids bigger gaps (bottom row, right image). Can you please explain why? $\endgroup$
    – user154385
    Commented Sep 12, 2023 at 14:37
  • $\begingroup$ Single Linkage merges in each step the two clusters that have the closest pair of points (e.g. the smallest gap). It does not matter how long-stretched the clusters are, it just looks at the gaps between the clusters. That leads to the long-stretched long blue and orange clusters on the bottom-right. $\endgroup$
    – Broele
    Commented Sep 12, 2023 at 17:41
  • $\begingroup$ Complete Linkage merges the two clusters that lead to the smallest diameter (diameter is the maximal distance between two points of the cluster), i.e. the resulting cluster should fit into a circle as small as possible. That probably leads to the weird outcome in the top-righ image. The right blue part will create a slightly bigger diameter if merged with the green part instead of merging it with the left blue part. $\endgroup$
    – Broele
    Commented Sep 12, 2023 at 17:45
  • 1
    $\begingroup$ I would not expect single linkage just to grow clusters, but also to merge clusters of equal size. Suggestion: create an own question for it and I will be happy to answer it and produce images to demonstrate what is going on $\endgroup$
    – Broele
    Commented Sep 13, 2023 at 9:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.