In word2vec I understand that selecting a vector size of lets say 100 would give me a word vector which has the correlation (kind of) between the word and 100 other words in corpus.
My question is are these 100 words same for each word?
$\begingroup$
$\endgroup$
8
-
$\begingroup$ no in general they are not the same. And the way to see this is to see the word2vec algorithm and/or implementation $\endgroup$– Nikos M.Oct 26, 2021 at 14:47
-
$\begingroup$ so if they are not same how does it make sense to average the word vectors to form a sentence? $\endgroup$– user16584277Oct 26, 2021 at 15:12
-
$\begingroup$ Can you elaborate on this averaging with some example? $\endgroup$– Nikos M.Oct 26, 2021 at 15:30
-
$\begingroup$ According to my understanding the context of a word is already some kind of average, so averaging over a sentence does not invalidate my previous claim. $\endgroup$– Nikos M.Oct 26, 2021 at 15:32
-
1$\begingroup$ A good explanation of word vectors is over here (including word2vec). Your confusion stems from the fact that you think that word vector dimensions actually represent other words, it is not like that. "Put differently, the weights that comprise a word vector are learned by making predictions on the probability that other words are contextually close to a given word." (from given ref) $\endgroup$– Nikos M.Oct 28, 2021 at 14:55
|
Show 3 more comments
2 Answers
$\begingroup$
$\endgroup$
The vector size is the number of dimensions in the embedding space. Each word in the vocabulary is represented by a vector. The vector size is the same for each word. The values in the vector are different for each word.
In your example, 100 is the vector size. The number of words is far larger, typically thousands or millions.
answered Jan 23, 2022 at 0:01
$\begingroup$
$\endgroup$
No, the 100 words that are selected as context words for each word in word2vec are not the same for every word. The context words are chosen based on their proximity to the target word in the training corpus. The idea is to capture the local context of each word, so the context words will vary depending on the specific context in which each word appears. This allows the word vectors to capture different aspects of meaning and relationships between words.
