2
$\begingroup$

When employing machine learning methods in NLP, most of studies use 200 or 300 dimensional vectors. 300 dimensional embeddings carry more information and this, therefore, is considered to produce better performance results in general.

If you have unlimited computational resources and the training time is not a problem for you, when does it make sense to use 200 dimensional embeddings instead of 300 dimensional vectors in a classification problem (e.g. in sentiment analysis) and why?

I am assuming that you are using off-the-shelf word2vec, GloVe, or other pretrained vectors. That is, the vectors are not being learnt from scratch in your classification task.

$\endgroup$
0
$\begingroup$

I recommend this paper. The authors treat the size of embeddings as a hyperparameter and provide a detailed study on it. They show that this dimensionality should depend on the corpus.

$\endgroup$
0
$\begingroup$

Dimensionality around word vectors is already answered here : https://stackoverflow.com/questions/45394949/what-is-dimensionality-in-word-embeddings

The reasons 200-300 dimensions are chosen normally, is that they have seen it produce the results that are very close to or equivalent to when they have chosen higher dimensions.

When it comes to training time, the time taken to generate word embeddings for 200-300 dimensions are not significantly different for most of the experiments I have personally come across.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.