How to alter word2vec wikipedia model for n-grams?

Question

I have a very little data, so my word2vec model does not perform well. My intention is to identify words similar to technical terms such as 'support vector machine', 'machine learning', 'artificial intelligence' etc.

I am interested in knowing if I can use the Google's wikipedia model for this. But according to my model most of the words I will be dealing with are n-grams. Hence, how can I utilise this Google's wikipedia model that is based on unigrams to achieve my task?

I am happy to provide more examples if needed :)

Answer 1

To clarify, an n-gram usually refers to a sequence of characters, i.e. the word "clear" is comprised of the trigrams {cle, lea, ear}. I think the term you are looking for is "multi-word phrases".

Embedding collections of words is referred to a couple different ways, including "sent2vec", "doc2vec" or "thought vectors". These terms generally refer to embedding a "complete" set of words, either a sentence, paragraph, collection of paragraphs, or a document. A common -- if somewhat inelegant -- approach to using a pre-trained word2vec model to embed multiple words is to embed each word separately and then take the average as the embedding for their combination.

I think you'll find this article relevant: Representation learning for very short texts using weighted word embedding aggregation

Answer 2

If you want to model the unique meaning of commonly occurring n-grams, often times called collocations, the best solution is to train a new embedding space to model those specific semantics.

Aggregating existing embeddings will only capture part of the collocation meaning.

If you do not have very much data, you can do fine-tuning/transfer learning. Fine-tuning/transfer learning takes an existing model architecture and weights, then does additionally training with more data. In this case, take Google's Wikipedia model and train it with your collocations. An example of fine-tuning for word2vec in Keras can be found here.