Find the correlation between two lists of texts

Question

Let's say that I have some lists of texts such as :

A = ["girl", "woman", "queen"]
B = ["boy", "man", "king"]
C = ["firefighter", "construction worker", "mechanic"]
D = ["nurse", "elementary school teacher", "esthetician"]

Can I calculate the correlations between the lists so that by the end I have a correlation matrix between every lists ?

The first obvious thing to do would be to apply embedding techniques such as BERT or Word2Vec on every lists but then what can I do ?

I would like something showing that A is correlated with D, B is correlated with C, A is negatively correlated with B etc

Answer 1

I see three options here :

• consider your list as a single string, for instance "girl woman queen" and embed this string using a model that is sensitive to the order of the words, like BERT, then your "correlation" would be the cosine similarity between embeddings. However, I expect this to behave weirdly since the resulting strings do not look like real sentences;
• compute two-by-two cosine similarities between embedded words, and take the mean, so for instance: $\frac{1}{3}\big{(}cosine(embed("girl"), embed("boy") + cosine(embed("woman"), embed("man")+ cosine(embed("queen"), embed("king")\big{)}$
• maybe the most "correlation"-like option: $corr(list_1, list_2) = \frac{\sum_i <embed(word_{i,1}),embed(word_{i,2})>}{\sum_i ||embed(word_{i,1})|| \sum_i ||embed(word_{i,2})||}$ where $<.,.>$ is the scalar product and $||.||$ is the corresponding norm.