1
$\begingroup$

I have 2 list of of sets and I want to calculate a distance.

set1 = [
  {'A', 'B', 'C'},
  {'A', 'D', 'X'},
  {'X', 'A'}
]

set2 = [
  {'A', 'B', 'C', 'D'},
  {'A', 'X'},
  {'X', 'A', 'B'}
]

So if the set of sets are equal I want the distance to be 0, and if unequal then I want the distance to be higher than 0.

The exact distance doesn't really matter as I'll ultimately be aggregating to compare multiple approaches to predicting this list of sets, so I really just need a relative distance.

My initial thought was a sum of Jaccard Distances, but I'm not sure how that would turn out.

$\endgroup$
3
  • $\begingroup$ Maybe 1 - sum([x == y for x,y in zip(set1,set2)])/max(len(set1),len(set2)) $\endgroup$
    – Multivac
    Apr 8 at 2:25
  • 1
    $\begingroup$ Will you always have the same amount of sets in both lists? $\endgroup$
    – Tasos
    Apr 8 at 9:38
  • $\begingroup$ @Tasos Yeah, the length of the lists will be the same. They are pairwise sets. $\endgroup$
    – pettinato
    Apr 8 at 15:52

1 Answer 1

1
$\begingroup$

Update

For pairwise comparison calculate each Jaccard distance and take the norm.

from numpy.linalg import norm

norm([ 1 - len(set.intersection(*p)) / len(set.union(*p)) for p in zip(set1,set2) ])
0.5335936864527374

OP

You can calculate the Jaccard distance.

With set1 and set2 in OP then

sc = list(map(lambda st: { ''.join(s) for s in st }, [set1, set2]))
1 - len(set.intersection(*sc)) / len(set.union(*sc))
0.8

Hope this helps.

$\endgroup$
3
  • $\begingroup$ I think this is basically Jaccard distance between set1 and set2, but I would really like a pairwise aggregation to draw out more subtlety. $\endgroup$
    – pettinato
    Apr 11 at 17:38
  • 1
    $\begingroup$ @pettinato See update for pairwise aggregation. $\endgroup$
    – Edmund
    Apr 11 at 18:07
  • $\begingroup$ A norm of some sort is a good idea, thanks! $\endgroup$
    – pettinato
    Apr 11 at 20:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.