How to train an existing word2vec gensim model on new words?

Question

According to gensim docs, you can take an existing word2vec model and further train it on new words.

The training is streamed, meaning sentences can be a generator, reading input data from disk on the fly, without loading the entire corpus into RAM.

It also means you can continue training the model later:

>>> model = Word2Vec.load("word2vec.model")
>>> model.train([["hello", "world"]], total_examples=1, epochs=1)

Source: docs

But when I actually try it, it doesn't seem to learn the new terms.

from gensim.models import Word2Vec

# initial a model
model = Word2Vec([["cat", "say", "meow"], ["dog", "say", "woof"]], min_count=1)

# count terms in model
print( len(model.wv.vocab) )

=> 5

# train existing model on new terms
model.train([['potoatoes', 'and', 'farmers']], total_examples=model.corpus_count, epochs=model.epochs)

# count terms in model
print( len(model.wv.vocab) )

=> 5

After adding new terms in the 2nd code block, the model still only has the same number of terms as before.

How can I make this work?

Answer 1

Add this line before training with the new terms.

model.build_vocab([['potoatoes', 'and', 'farmers']], update=True)

After training,

print(len(model.wv.vocab))
=> 8

Answer 2

I think you cannot sort vocabulary after model weights already initialized.In your code you try to diplay the length of your vocabulary"print( len(model.wv.vocab) )" it is normal that it won't change, because you built your vocabulary before training your model and it wasn't changed.

Answer 3

I think your problem is in the value you're giving to the argument total_examples.

Trying giving it: total_examples = 1, which is the number of sentences you intend to train.

from gensim.models import Word2Vec 

docs = [["cat", "say", "meow"], ["dog", "say", "woof"]]     
model = Word2Vec(common_texts, size = 100, window = 5, min_count = 1, workers = 4)

docs = [['potoatoes', 'and', 'farmers']]    
model.train(docs, total_examples = len(docs), epochs = 10)

Answer 4

Today I ran into the same problem and I fixed it by setting model.min_count = 1.

So in my case the complete code is

   def train_sentences(self, sentences: List[List[str]], epochs: int = 1) -> None:
        self.model.min_count = 1  # so even words that only appears once are used
        self.model.build_vocab(sentences=sentences, update=True)  # update = True ensures that words are added to vocab
        self.model.train(sentences=sentences, epochs=epochs, total_examples=len(sentences))

Hope it helps somebody to save some time.