3
$\begingroup$

how much data does word2vec require? Are there any public data sets that are useful?

For example, could it be that 1000 newspaper articles are enough to use word2vec?

Here is a word2vec tutorial from Kaggle that uses 50,000 movie reviews. I am trying to understand the scale of the word2vec input.

$\endgroup$
2
$\begingroup$

As word2vec is a neural network, it benefits from very large datasets. The Kaggle dataset is 50,000 reviews * ~5 sentences per review, so about a quarter million sentences. As they note, they get approximately the same results using bag of words and word2vec. One thing which is of note, since the review data comes from the internet, the sentences are much more loosely structured than what you would encounter in a corpus of newspapers, which typically go through grammatical review. A great dataset for training word2vec on structured language is the wikipedia dataset: https://en.wikipedia.org/wiki/Wikipedia:Database_download

$\endgroup$
  • $\begingroup$ I am not the OP, but wondering, do you know if it is possible to apply data augmentation to text in order to extend the dataset? (as is done to images when training CNNs) $\endgroup$ – Ricardo Cruz Jul 29 '16 at 12:03
  • $\begingroup$ It stands to reason that it ought work, but I've never done this myself. In the case of the IMDB dataset, you might try using the Yelp dataset to expand the language base; it's still text associated with a rating, so it may increase your flexibility to 'sarcasm' words. $\endgroup$ – j.a.gartner Jul 29 '16 at 15:46
1
$\begingroup$

You can download already generated vectors from

FastText https://fasttext.cc/docs/en/english-vectors.html for Wiki + some other web pages

and

SpaCy https://spacy.io/models/ - for Common Crawl

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.