2
$\begingroup$

I am using GloVe and gensim for my project. I have a corpus of data (let's say mydata.txt) which has new words which are not in the existing Glove. So, how do I retrain glove so that the existing pre-trained glove must now include the new words on my corpus mydata.txt? I have been struggling and failed to find the solution for 2 weeks. The only similar post I found is this Improving existing GloVe Model

$\endgroup$
  • 1
    $\begingroup$ Did you try the suggested solution that you linked to? $\endgroup$ – Tom M. Jun 29 '18 at 12:46
  • $\begingroup$ Please tell us what code you use, there is no implementation of GloVe in Gensim. $\endgroup$ – Robin Jun 29 '18 at 13:23
1
$\begingroup$

What you should do is:

  • Create a new instance of a GloVe model with the old_words and new_words as vocabulary.
  • Replace the initial vectors/biases of the old_words with the ones you have already.
  • Train this model on mydata.txt.

The new old_words representations won't be the same but will be highly influenced by the old ones.

| improve this answer | |
$\endgroup$
1
$\begingroup$

I am not sure if this is the best method, but this is how I did it. You cannot call it retraining, but this is one of the way you can add on your data with the glove vectors.

from gensim.models import Word2Vec

glove_6b = "glove.6B.100d.txt"

#loading the glove vectors 

with open(glove_6b, "rb") as lines:
     wvec = {
        line.split()[0].decode(encoding): np.array(line.split()[1:], 
                                                         dtype=np.float32)
                                                         for line in lines}

#my data vectors

em_model = Word2Vec(text_data2, size=100, window=5, min_count=1, workers=2)
w2v = {w: vec for w, vec in zip(em_model.wv.index2word, em_model.wv.vectors)}

a = list(w2v.keys())

#mixing them both 
for i in a:
    if i in wvec:
       continue
    else:
       wvec.update({ i  : w2v[i]})

len(list(wvec.keys()))

surely, can be improved. hope this helps.

| improve this answer | |
$\endgroup$
0
$\begingroup$

I don't think the link in the question, nor the idea of adding vectors together are viable.

I believe GloVe (Global Vectors) is not meant to be appended, since it is based on the corpus' overall word co-occurrence statistics from a single corpus known only at initial training time

What you can do is use gensim.scripts.glove2word2vec api to convert GloVe vectors into word2vec, but I don't think you can continue training since its loading in a KeyedVector not a Full Model

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.