# Help me understand how word-as-vector representations are constructed

Let's suppose I have a big list of words. I want to turn this list into a vector space of dimension $N$ such that each word is a vector in this vector space. But I have no idea how to go about with that. Some questions:

1. Is the list enough? For each element of the list, do I need $x$ example sentences also?
2. How does the computer deduce the dimensions of the vector space from the list/corpus?
3. Is there a way to figure out whether the dimensions of the vector space correspond to something in English?

## 1 Answer

I am assuming that you mean a vector representation of words, not to be confused by the vector representation produced in a bag of words approach that represent a document in vector space. Word2vec is an approach in which you train a model to represent words as a function of the provided context.

The answers that follow are:

• 1) No: You'll need some representation of context in which the word is used. For example: Skip-grams.
• 2) No: That is a user defined parameter
• 3) < I do not understand this question >
• Thanks. For #3, let's suppose I have a vocabulary of the words {king, queen, man, woman}. The algorithm then constructs a vector space (ROYALTY, MASCULINE, FEMININE). So what I'm asking is, is the algorithm powerful enough to find the basis elements of the vector space? – BalancedTryteOperators Feb 20 '18 at 20:37
• That is not how it works: You'll end up with a model that would convert any word of the vocabulary to a vector of N values between 0 and 1. Since the model is ordered by context found around the words, and we assume that that context is dependent on semantics, semantic relations are encoded in the ordering. Hence you could, for instance, practically search the space for the closest keys: this should yield the most similar words. Perhaps this link is helpful: radimrehurek.com/gensim/models/word2vec.html – S van Balen Feb 20 '18 at 21:19
• Thanks again. However, I was reading an article that said word2vec can perform vector addition and subtraction on words. So for instance "king - man + woman" would return the answer you would expect ("queen"). Is that true? – BalancedTryteOperators Feb 20 '18 at 23:01
• Yes and no: You can perform operations on those resulting (numerical) vectors, just like you describe. The resulting coordinates can that be coded back to words. However the examples from literature seem to be well chosen (as in: those particular examples work well) – S van Balen Feb 21 '18 at 7:33