When training a word2vec model with, eg, gensim, you can specify the minimum times a word needs to be seen (with the parameter min_count). The default value for this seems to be 5.

Are there any theoretical considerations for selecting a threshold for min_value? Depending on the contexts the words are seen in (and the potential variety of them), it seems that it could take more than 5 words for the model to learn a good vector representation of the word. Are there any papers that specify when the learned vectors become more static? Alternatively, can you train the model on all words, but only utilize the vectors of words that appeared in the corpus more than a specified number of times? Is frequency a better cutoff than min_count?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.