I'm wondering how Word2Vec is being constructed.
I've read tutorials simply stating that we can train a skip grams neural network model and use the weights that are trained as word vectors.
However, I've also seen this picture:
If reading this diagram correctly:
1) The CBOW and Skip Grams model are both trained with some inputs
2) The output of CBOW is used as an input to the middle neural network
3) The output of skip grams is used as an output to the middle neural network.
The output of CBOW is the a prediction of the center word given a context, and the output of skip grams is the prediction of the surrounding center word.
These outputs are then used to train another set of neural network.
Hence we first train the CBOW, and Skip-gram neural network, then train the middle neural network afterwards? And the input to the middle neural network is one hot encoded.
Is the above interpretation correct?