From this article:
- In vanilla skip gram model, softmax is computationally very expensive, as it requires scanning through the entire output embedding matrix (W_output) to compute the probability distribution of all V words, where V can be millions or more.
- Furtheremore, the normalization factor in the denominator also requires V iterations.
Hence the article suggests applying negative sampling instead of softmax. There are many article discussing skip grams with negative sampling. But I did not find any discussing CBOW (Continuous Bag of Words Model ) model with negative sampling. Why is this so? Is it not possible / recommended? Or its exacty same as skip gram? Can you please shed some insights about using negative sampling with CBOW?
PS: any article / paper link will also be of great help.