0
$\begingroup$

I've been reading about consensus clustering and the consensus matrix in this article. I understand how the consensus matrix is made after re-sampling and clustering parts of your data H times. I understand that the consensus matrix is used to determine the optimal amount of clusters (k) and allows for making a nice heatmap. What I don't understand is how this consensus matrix results in your final clustering.

Say I have a consensus matrix of 4*4 (so we have 4 items to be clustered) where each value between 0 and 1 in the matrix represents the number of times items i and j are assigned to the same cluster, divided by the total numer of times both items are selected for clustering. We could have the following consensus matrix after 4 iterations of 80% sub-samples (taken from here). We kept track of all 4 clusters that were made in the process of obtaining this consensus matrix. How do we select the final clustering based on this consensus matrix?

enter image description here

Improve this question
$\endgroup$
5
  • $\begingroup$ What is the "consensus index value" on that distribution plot in the original post? $\endgroup$
    – WBM
    Feb 26, 2021 at 10:34
  • $\begingroup$ @WBM that's the values we see in the image above. In the most ideal scenario we'd have only 0's and 1's which indicates perfect consensus. $\endgroup$
    – Sandertjuhh
    Feb 26, 2021 at 10:36
  • 1
    $\begingroup$ So isn't this then an additional evaluation technique (i.e. how much consensus is there between clusters), rather than explicit clustering? The clusters would drop out earlier in the processing. It looks like you use the consensus metric to test for the best K value $\endgroup$
    – WBM
    Feb 26, 2021 at 10:43
  • $\begingroup$ @WBM Yes, maybe you're right and it's not meant for deciding on the final clustering. It would then only give more certainty on the value of k and the stability of the discovered clusters. And then after you gained more insight, you can cluster again using a regular clustering technique with the found value of k? $\endgroup$
    – Sandertjuhh
    Feb 26, 2021 at 10:48
  • $\begingroup$ I believe so, much like the Silhouette score $\endgroup$
    – WBM
    Feb 26, 2021 at 10:49

1 Answer 1

Reset to default
1
$\begingroup$

Consensus clustering is an additional evaluation technique (i.e. how much consensus is there between clusters), like the Silhouette score, rather than explicit clustering.

The clusters themselves would drop out earlier in the processing. The consensus metric is used to test for the best K value, in the K-means clustering algorithm.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Let me know if you have any questions otherwise please accept this answer $\endgroup$
    – WBM
    Mar 4, 2021 at 10:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.