0
$\begingroup$

I have data of the format [dataset_N, dataset_N+1, similarity] where two different datasets have been compared, resulting in a similarity score. A score is generated for each possible (unique) pair of datasets (~7e6 pairs).

I am trying to determine how to cluster these datasets using just the similarity scores.

Something like K-means can't be used directly because the available information (similarity scores and which pair was compared) doesn't match its inputs.

Is there an existing method that can be used for this?

Edit: reworded for clarity

Edit 2: removed references to labels, since that was causing confusion

Improve this question
$\endgroup$
7
  • $\begingroup$ Show a few lines of the dataset itself and it might be easier to understand what you are asking $\endgroup$
    – sconfluentus
    Commented Oct 7, 2022 at 3:09
  • $\begingroup$ What do you mean by different label sets? I am a little confused since you use the word label to describe your variables. Are you trying to cluster the samples? Also when you say "it interprets the labels as location coordinates," what is "it"? Are you using python, R , matlab or something else? why would k-means interpret the labels as location coordinates? Can you post your code and/or some of your dataset? Is this a biological dataset? Like @sconfluentus said, it will be easier to answer your question if you provide more detail $\endgroup$
    – fractalnature
    Commented Oct 7, 2022 at 19:19
  • $\begingroup$ I've reworded the question. What I meant by "label sets" is that the clusters would group the dataset labels together based on dataset similarity, resulting in a set of labels per cluster. I'm using Python, though I assumed most common algorithms have been implemented in multiple languages, so that was a secondary concern. The datasets have already been compared and scores generated, so in theory they are not relevant for this part of the analysis. $\endgroup$
    – Mandias
    Commented Oct 8, 2022 at 22:12
  • $\begingroup$ Why do you think that KNN does not work with 3 features? $\endgroup$
    – liakoyras
    Commented Oct 9, 2022 at 17:28
  • $\begingroup$ The labels are not appropriate inputs because they just identify the dataset. Each is unique, so not appropriate for spatial clustering. $\endgroup$
    – Mandias
    Commented Oct 10, 2022 at 18:37

1 Answer 1

Reset to default
0
$\begingroup$

Thanks to the clues from the comments and this stack exchange post outlining graph clustering and modularity scores I was able to find a suitable method. I went with the NetworkX implementation of the Louvain Communities algorithm.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.