2
$\begingroup$

A CBOW model actually takes multiple words as inputs and a targeted central word as the output.
So, the trained model actually maps several words to a single one, I mean it takes context words and outputs the central word. But what we expected to get is model mapping a word to its vector representation. It seems the output is coincident but not the input and the mapping.

So like in genism, how it really works to map a word to its vector representation? Does it just save all final model's outputs as the central words' vector representation? But the final model's outputs would close to the ground truth's one-hot embedding rather than a vector with context information.

For short, my question is:
How does a CBoW model convert one word to its vector representation?

$\endgroup$
0
$\begingroup$

CBoW (Continuous bag-of-words) is a theoretical architecture, not exactly a saved model or a library like gensim. Gensim might be an implementation of CBoW to which you feed a one-hot vector and get your word vector output.

CBoW as a theoretical model gives 'representational meaning' to a word based on the 'representational meaning' of the word surrounding it. 'Representational meaning' may just be a fancy term for word vectors. So basically the model builds word vectors out of surrounding word vectors possibly using negative sampling and the NCE loss, but the main point is that it fine-tunes these vectors until their 'representational meaning' becomes well-honed.

These well honed vectors are then probably callable from gensim through one-hot inputs.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.