3
$\begingroup$

I'm using GloVe pre-trained word vectors (glove.6b.50d.txt, glove.6b.300d.txt) as word embedding.

I have a conceptual question:

  • What is the difference between these files?
  • On the other hand, what does the dimension represent in the GloVe pre-trained word vectors?
$\endgroup$
2
  • $\begingroup$ I'm voting to close this question as off-topic because the community is not for questions which try to ask the difference between files. StackOverflow could be a better option. $\endgroup$ – Shubham Panchal Oct 14 '19 at 6:30
  • $\begingroup$ @ShubhamPanchal What about the second question? $\endgroup$ – Benyamin Jafari Oct 14 '19 at 6:51
3
$\begingroup$

Glove creates word vectors that capture meaning in vector space by taking global count statistics. The training objective of GloVe is to learn word vectors such that their dot product equals the logarithm of the words probability of co-occurrence. while optimizing this, you can use any number of hidden representations for word vector. In the original paper, they trained with 25, 50, 100, 200, 300. These dimensions are not interpretable. after training, we are getting a vector with 'd' dim that captures many properties of that word. If the dimension is increasing, the vector can capture much more information but computational complexity will also increase. please go through this blog.

$\endgroup$
3
  • $\begingroup$ Thanks for the valued response. $\endgroup$ – Benyamin Jafari Oct 14 '19 at 10:37
  • $\begingroup$ What does "hey trained with 25, 50, 100, 200, 300" mean? What is 50 ... ? $\endgroup$ – abdoulsn Jun 1 '20 at 0:21
  • $\begingroup$ @abdoulsn there the numbers represent the size of the vectors in the vocabulary. In a 50 dimensional vocabulary tokens are represented as 50-dimensional vectors, in a 100-dimensional vocabulary tokens are represented as 100-dimensional vectors, etc. $\endgroup$ – Oso Aug 27 '20 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.