what actually word embedding dimensions values represent?

Question

I am learning word2vec and word embedding , I have downloaded GloVe pre-trained word embedding (shape 40,000 x 50) and using this function to extract information from that:

import numpy as np
def loadGloveModel(gloveFile):
    print ("Loading Glove Model")
    f = open(gloveFile,'r')
    model = {}
    for line in f:
        splitLine = line.split()
        word = splitLine[0]
        embedding = np.array([float(val) for val in splitLine[1:]])
        model[word] = embedding
    print ("Done.",len(model)," words loaded!")
    return model

Now if I call this function for word 'hello' something like :

print(loadGloveModel('glove.6B.100d.txt')['hello'])

it gives me 1x50 shape vector like this:

[ 0.26688    0.39632    0.6169    -0.77451   -0.1039     0.26697
  0.2788     0.30992    0.0054685 -0.085256   0.73602   -0.098432
  0.5479    -0.030305   0.33479    0.14094   -0.0070003  0.32569
  0.22902    0.46557   -0.19531    0.37491   -0.7139    -0.51775
  0.77039    1.0881    -0.66011   -0.16234    0.9119     0.21046
  0.047494   1.0019     1.1133     0.70094   -0.08696    0.47571
  0.1636    -0.44469    0.4469    -0.93817    0.013101   0.085964
 -0.67456    0.49662   -0.037827  -0.11038   -0.28612    0.074606
 -0.31527   -0.093774  -0.57069    0.66865    0.45307   -0.34154
 -0.7166    -0.75273    0.075212   0.57903   -0.1191    -0.11379
 -0.10026    0.71341   -1.1574    -0.74026    0.40452    0.18023
  0.21449    0.37638    0.11239   -0.53639   -0.025092   0.31886
 -0.25013   -0.63283   -0.011843   1.377      0.86013    0.20476
 -0.36815   -0.68874    0.53512   -0.46556    0.27389    0.4118
 -0.854     -0.046288   0.11304   -0.27326    0.15636   -0.20334
  0.53586    0.59784    0.60469    0.13735    0.42232   -0.61279
 -0.38486    0.35842   -0.48464    0.30728  ]

Now I am not getting what actually these values represent , ( I know its result of hidden layer of single layer neural network ) but my confusion is what actually these weights represent and how it is useful for me?

Because what I was getting suppose if I have :

Here I understand because each word is mapping to each column category label, But in the GloVe there are no columns labels for 50 columns, it just returns 50 values vector, so what actually these vectors represent and what i can do with it? I am trying to find this since 4-5 hours but everyone/every tutorial on the internet explaining what are word embedding and how they looks like but no one explaining what actually these weights represent?

Answer 1

These columns are actually arbitrary, they do not represent anything for humans. However, it does not mean they are useless, quite the opposite - computers can extract features from this highly dimensional space more easily, such as in neural networks.

To extract information which is useful for humans, I recommend to have a look at examples in original paper by Tomas Mikolov:

We can have a look at common capital city. Athens is to Greece (in sense that they have similar relationship) as is Oslo to Norway. We can use afterwards vectors obtained using GloVe and construct

$$v(Athens) - v(Greece) + v(Norway)$$

and this vector in 'globally' (meaning that trained on universal dataset, such as wikipedia) trained model would be closest to $v(Oslo)$.