How does a CBoW model convert a word to a vector?

Question

A CBOW model actually takes multiple words as inputs and a targeted central word as the output.
So, the trained model actually maps several words to a single one, I mean it takes context words and outputs the central word. But what we expected to get is model mapping a word to its vector representation. It seems the output is coincident but not the input and the mapping.

So like in genism, how it really works to map a word to its vector representation? Does it just save all final model's outputs as the central words' vector representation? But the final model's outputs would close to the ground truth's one-hot embedding rather than a vector with context information.

For short, my question is:
How does a CBoW model convert one word to its vector representation?

Answer 1

CBoW (Continuous bag-of-words) is a theoretical architecture, not exactly a saved model or a library like gensim. Gensim might be an implementation of CBoW to which you feed a one-hot vector and get your word vector output.

CBoW as a theoretical model gives 'representational meaning' to a word based on the 'representational meaning' of the word surrounding it. 'Representational meaning' may just be a fancy term for word vectors. So basically the model builds word vectors out of surrounding word vectors possibly using negative sampling and the NCE loss, but the main point is that it fine-tunes these vectors until their 'representational meaning' becomes well-honed.

These well honed vectors are then probably callable from gensim through one-hot inputs.

Answer 2

CBOW while training tries to predict the main word from the context words.

Once the training is done we use the weight matrix of first hidden layer to get Word2vec embedding. The weight matrix is multiplies by the one hot encoding to get the word2vec emebdding.

If you multiply a 1 x 10,000 one-hot vector by a 10,000 x 300 matrix, it will effectively just select the matrix row corresponding to the “1”. Here’s a small example to give you a visual. :

This means that the hidden layer of this model is really just operating as a lookup table. The output of the hidden layer is just the “word vector” for the input word.

References : http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/