# can I use public pretrained word2vec, and continue train it for domain specific text?

I have a set of reviews from apparel domain, about 100K reviews (2M words). And I want to train word2vec to do some cool NLP staff with it.

However the size is not enough for creating adequate word2vec model, it requires billions of words.

So the idea is to use public corpora (e.g. wikipedia), or even use some pretrained models (e.g. from gensim cool framework) and add my domain specific text. I assume the model will get aware of unseen-in-public words, and could correct vectors for common word.

Does it make sense ? will the 2M words makes any effect at all ?

• Yes, you can. Welcome to the site. – Emre Aug 21 '18 at 17:07
• Welcome to the site. The 100k reviews is not a small data. word2vec does not require billions of words to work well. You can start training on your own dataset first and check how it performs. – hssay Aug 23 '18 at 1:52

Yes, you can fine-tune your embeddings while using pre-trained word vectors. If you are using tensorflow, in your tf.get_variable parameter, set trainable=True. The reason it will work is that the only 2M words will be taken from the pre-trained embeddings. Those embeddings will already be close to a local optimum and can be further optimized in very few iterations.