I have a precomputed distance matrix.

I'm trying to do an hierarchical clustering using scipy:

from scipy.cluster.hierarchy import fcluster, linkage, cut_tree,fcluster

z = linkage(ds, "average")
clust = fcluster(z, 4, criterion='maxclust')

The problem is that, most of the labels ends up in one group - Why?

By the way, I tried to use AgglomerativeClustering as well but I got similar results.

ds is the precomputed distance matrix. I'm using DTW, a distance metric for time series analysis.

I uploaded ds.csv to here.

Also, here's an heatmap which forms interesting lines (I don't really know how to interpret this currently)

enter image description here

  • $\begingroup$ As far as I remember this can be related to the specific method used to merge instances; did you try another value instead of criterion='maxclust'? $\endgroup$
    – Erwan
    Oct 22 '19 at 16:56

Likely your precomputed distance matrix is bad.

Judging from your plot, it's probably a half-filled similarity matrix instead. And it likely is not in the required form for linkage.

The input y may be either a 1d condensed distance matrix or a 2d array of observation vectors.

Did you make your distance matrix "condensed"? Otherwise it will be treated as data matrix, not distance matrix, because python does not have type safety but uses duck typing - if it quacks like a data matrix, it is a data matrix. Then you end up computing Euclidean distances on half-filled rows of this matrix...


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.