1
$\begingroup$

I am totally new to this topic, that's why I am so confused or stuck in this code for a while, but I am not sure how to solve it correctly. My goal is to write a short text embedding using vector representation from the text. The word embeddings are aggregated via mean averaging to infer a vector representation for the text. I generated model vectors using gensim.models and then I run each through the model and check if the word is inside it. If yes, I will embed it and then aggregate the mean average ( not sure if is correct). After that, I want to compare it with cosine similarity, but I am not sure how.

from sklearn.metrics.pairwise import cosine_similarity

first_sentence_list = ['driver', 'backs', 'into', 'stroller', 'with', 'child', ',', 'drives', 'off']
second_sentence_list = ['driver', 'backs', 'into', 'mom', ',', 'stroller', 'with', 'child', 'then', 'drives', 'off']

//

def meanEmbeddings(text_list):
    model = load_wiki_en_vectors()

    test = []
//loop the given sentence
    for word in text_list:
        try:
            word_embeding = model.get_vector(word, norm=True)
            test.append(np.mean(word_embeding,axis=0)) // not sure if this is right doing mean averaging here
        except KeyError:
            continue
    return test


res_1 = meanEmbeddings(first_sentence)
//[0.0023045307, 0.0033775743, ...]
res_2 = meanEmbeddings(second_sentence)
//[0.0023045307, 0.0033775743,...]

After want to do the similarity check using sklearn pairwise cosine similarity library. The problem is here, I have two different length of ( first 9 and second 11)


cos = cosine_similarity([res_1],[res_2])
$\endgroup$

2 Answers 2

0
$\begingroup$

This should work, you dont need to append.

   def meanEmbeddings(model, words):
        # remove out-of-vocabulary words
        words = [word for word in words if word in model.vocab]
        if len(words) >= 1:
            return np.mean(model[words], axis=0)
        else:
            return []
$\endgroup$
0
$\begingroup$

Try this. This will average the word embeddings to obtain the sentence embeddings.

# import libs
import gensim
from sklearn.metrics.pairwise import cosine_similarity

# load word2vec
model = gensim.models.KeyedVectors.load_word2vec_format('path to word2vec e.g. GoogleNews-vectors-negative300.bin', binary=True)

# your inputs
first_sentence_list = ['driver', 'backs', 'into', 'stroller', 'with', 'child', ',', 'drives', 'off']
second_sentence_list = ['driver', 'backs', 'into', 'mom', ',', 'stroller', 'with', 'child', 'then', 'drives', 'off']

# remove oov
first = [word for word in first_sentence_list if word in model.key_to_index]
second = [word for word in second_sentence_list if word in model.key_to_index]

# average word embeddings to get sentence embeddings
first_sent_embedding = np.mean(model[first], axis=0)
second_sent_embedding = np.mean(model[second], axis=0)

# calculate similarities
result = cosine_similarity(first_sent_embedding.reshape(1,-1),second_sent_embedding.reshape(1,-1))

print(result)
$\endgroup$
2
  • 1
    $\begingroup$ Thanks for the answer. Just a quick correction: you have two first_sent_embeddings under the "# calculate similarities" in the result variable. One of them should be second_sent_embeddings. $\endgroup$ Mar 5 at 17:04
  • $\begingroup$ Thanks. I just corrected it. You could do that too : ) $\endgroup$
    – Abu Shoeb
    Mar 6 at 5:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.