10
$\begingroup$

I trained word embeddings with 300 dimensions. Now, I would like to have word embeddings with 50 dimensions: is it better to retrain the word embeddings with 50 dimensions, or can I use some dimensionality reduction method to scale the word embeddings with 300 dimensions down to 50 dimensions?

Improve this question
$\endgroup$
2
  • $\begingroup$ what method of word embedding are you using? $\endgroup$
    – lollercoaster
    Jul 28, 2015 at 20:23
  • $\begingroup$ @lollercoaster word2vec and GloVe. $\endgroup$
    – Franck Dernoncourt
    Jul 28, 2015 at 20:26

2 Answers 2

Reset to default
6
$\begingroup$

There is a paper on this subject calle

Simple and Effective Dimensionality Reduction for Word Embeddings, Vikas Raunak

You can read it here

You can also find the implementation here

In my opinion it works quite well

Improve this answer
$\endgroup$
4
$\begingroup$

t-distributed stochastic neighbor embedding (t-SNE) is often used for dimensionality reduction in word embeddings. t-SNE maintains the relative relationships between the vectors.

Most often t-SNE is used for visualization, thus reducing the dimensions to 2 or 3. It could also reduce the dimensions down to 50.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.