0
$\begingroup$

I am working on a clustering problem. I am not able to find the right similarity metric for my system.

I have n nodes with ordered vector (eg: [1,0,0,0,1,0,.....] "1" represents represent presence of ith object and "0" represents absence of the object). I want to cluster them.

$\endgroup$
2
$\begingroup$

Cosine is for continuous values. It's not the most appropriate thing here.

For binary values, look at

  • Simple matching distance
  • Hamming distance
  • Jaccard distance
  • Many many many more.

Don't expect clustering to just work. If you have binary data, you only have very few bits of information. The resulting distances will show discrete steps, and clustering will likely still fail to provide good results. Roughly, you have "identical", "one difference", "two differences", and then probably everything already is connected. It's because of the data, and a not very well defined task (what would be a good cluster?).

$\endgroup$
1
$\begingroup$

Cosine similarity is a popular choice: https://en.wikipedia.org/wiki/Cosine_similarity

$\endgroup$
1
$\begingroup$

As you can imagine, the answer is "it depends".

But in your case you can choose between two: Euclidean or Cosine.

As the dimensionality grows every point approach the border of the multi dimensional space where they lie, so the Euclidean distances between points tends asymptotically to be the same, which in similarity terms means that the points are all very similar to each other.

So, in those cases, you go for the cosine similarity; let's say for dimensionality bigger than 100.

Otherwise you can go for the traditional, Euclidean similarity.

$\endgroup$
0
$\begingroup$

I would strongly recommend to also have a look at Graph Theoretic measures as your problem can be easily and properly seen as a Bipartite-Graph Clustering.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.