-1
$\begingroup$

I have a list of job titles. I found the semantic similarity between them by using word2vec in spacy.

Now I want job titles which have more than 83% similarity be in the same cluster. For example I have:

titles=[art teacher, gym teacher, basketball teacher, painting teacher]
art_teacher=[1, 0.7,0.6,0.91] 
gym_teacher=[0.7,1, 0.9,0.5]
basketball_teacher=[0.6, 0.9,1,0.55]
painting_teacher=[0.91,0.5,0.55,1]

I want names that have more than 85% similarity to be clustered together, so we would have:

cluster1: art teacher , painting teacher

cluster2: basketball teacher, gym teacher

$\endgroup$
1
  • $\begingroup$ What have you tried so far? $\endgroup$
    – WBM
    Commented Feb 24, 2021 at 15:41

1 Answer 1

0
$\begingroup$

There might already be a built-in function to compare these outputs you've shown, but one solution would be to just threshold the lists into Boolean lists, and then use logical_and to compare them:

import numpy as np

def threshold_clusters(teacher_list, threshold = 0.85):
    return [i>threshold for i in teacher_list]

def compare_clusters(first,second):
    first = threshold_clusters(first)
    second = threshold_clusters(second)
    return np.logical_and(first, second).any()

print(compare_clusters(teacher,gym_teacher))
print(compare_clusters(teacher,basketball_teacher))
print(compare_clusters(teacher,painting_teacher))
print(compare_clusters(gym_teacher,basketball_teacher))
print(compare_clusters(gym_teacher,painting_teacher))
print(compare_clusters(basketball_teacher,painting_teacher))

Output:

False
False
True
True
False
False
$\endgroup$
2
  • $\begingroup$ Did this solve your problem? Let me know if you have any questions $\endgroup$
    – WBM
    Commented Feb 26, 2021 at 15:32
  • $\begingroup$ Otherwise please accept the answer $\endgroup$
    – WBM
    Commented Mar 4, 2021 at 10:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.