I'm working with domain-oriented documents in order to obtain synonyms using Word2Vec. These documents are usually templates, so sentences are repeated a lot.
1k of the unique sentences represent 83% of the text corpus; while 41k of the unique sentences represent the remaining 17% of the corpus.
Can this unbalance in sentence frequency impact my results? Should I sub-sample the most frequent sentences?
Are the sentences exactly the same, word to word? If that is the case I would suggest removing the repeated sentences because that might create a bias for the word2vec model, ie. repeating the same sentence would overweigh those examples single would end with higher frequency of these words in the model. But it might be the case that this works in your favor for find synonyms. Subsample all the unique sentences, not just the most frequent ones to have a balanced model.
I would also suggest looking at the FastText model which is built on top of the word2vec model, builds n grams at a character level. It is easy to train using gensim and has some prebuilt functions like model.most_similar(positive=[word],topn=number of matching words) to find the nearest neighbors based on the word embeddings, you can also use model.similarity(word1, word2) to easily get a similarity score between 0 and 1. These functions might be helpful to find synonyms.
