Word2Vec: Using pre-trained models

Question

I am unable to find details for pretrained word2vec models. If someone is about to use a pre-trained model should it be clear what kind of pre-processing was done before the model was trained? Such as if lower cased was applied or removal of specific stopwords?

• Glove: Is not referring to any kind of that information except of recommending stanford tokenizer (splitting words).
• Google pre trained model: Doesn't give any info for pre-processing of the pretrained model. Refers in a sentence to use a script for the wikipedia training data, but do not mention how the google news were preprocessed before trained.

And in other cases of pretrained models I cant find this kind of information. Someone could assume that there is a specific way of pre-processing for word2vec but I remember that the instructor of stanford NLP course, Richard Socher, that some people are removing stopwords and some others do not.

I have also noticed that people are using different pre-processing methodology and same pre-trained model for the kaggle competition Bag of Words Meets Bags of Popcorn which doesnt make sense for me.

Any one can give some insights?

Answer 1

I think in a lot of cases, when people are using pre-trained models, they force their words to lower and singular case and then test to see if the lowercase and singular (not plural) version were trained. Sometimes, you have to check yourself like A.M. Bittlingmayer said.

To provide some insight into why sometimes people remove stopwords versus not --- I think it has to do with whether they want to value infrequent words more or less. Because you don't want to value too highly very frequent words, the algorithm 'punishes' extremely frequent words. So, one could argue that because this already happens, you don't need to remove stopwords since they will be 'punished' anyways. So, not everyone removes stopwords.