4
$\begingroup$

how much data does word2vec require? Are there any public data sets that are useful?

For example, could it be that 1000 newspaper articles are enough to use word2vec?

Here is a word2vec tutorial from Kaggle that uses 50,000 movie reviews. I am trying to understand the scale of the word2vec input.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
3
$\begingroup$

As word2vec is a neural network, it benefits from very large datasets. The Kaggle dataset is 50,000 reviews * ~5 sentences per review, so about a quarter million sentences. As they note, they get approximately the same results using bag of words and word2vec. One thing which is of note, since the review data comes from the internet, the sentences are much more loosely structured than what you would encounter in a corpus of newspapers, which typically go through grammatical review. A great dataset for training word2vec on structured language is the wikipedia dataset: https://en.wikipedia.org/wiki/Wikipedia:Database_download

Improve this answer
$\endgroup$
2
  • $\begingroup$ I am not the OP, but wondering, do you know if it is possible to apply data augmentation to text in order to extend the dataset? (as is done to images when training CNNs) $\endgroup$
    – Ricardo Cruz
    Commented Jul 29, 2016 at 12:03
  • $\begingroup$ It stands to reason that it ought work, but I've never done this myself. In the case of the IMDB dataset, you might try using the Yelp dataset to expand the language base; it's still text associated with a rating, so it may increase your flexibility to 'sarcasm' words. $\endgroup$
    – j.a.gartner
    Commented Jul 29, 2016 at 15:46
1
$\begingroup$

You can download already generated vectors from

FastText https://fasttext.cc/docs/en/english-vectors.html for Wiki + some other web pages

and

SpaCy https://spacy.io/models/ - for Common Crawl

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.