Let's take the CBOW (continuous bag of words) model as the example.
Suppose that, there are $c$ context words, each of which is a one-hot encoding vector. So the total number of elements of input vector is $v \cdot c$, where $v$ is the total number of words to be learned.
Suppose the first hidden layer has $d$ neurons, which is the size of the embedding dimension.
So all weights (or: embedding) parameters that are to be learned can be arranged in a $v \times d$ matrix.
My question is concerning how many weight parameters are updated during each training epoch for the word2vec model. Are all the $v \cdot d$ embedding, parameters updated during each training epoch? Or, are only $c \cdot d$ of them updated during each training epoch? Here the $c \cdot d$ embedding parameters are for the input context words.
If all $v \cdot d$ weights got updated during each epoch, it would be much more expensive, considering that the typical value of $v$ ranges from tens of thousands to hundreds of thousands.