# How is CBOW different from building PMI matrix and then reducing using PCA

PMI matrix and reduction using PCA: Based on the number of times 2 words appear together (in a certain pre-defined window), and the individual frequency of words, we build the PMI matrix. Then reduce it using PCA, to get dense representations of each word in the corpus, which are able to capture some semantics of the text

CBOW: Learning word representations through a neural network, whose end objective is to maximize the probability of correct word pairs. The probability values are known in advance by counting the number of times a word is appearing in another word's context in the training data.

Both of these methods are using counts, and then getting a dense word representation. Is there a definitive advantage of one over the other? Why was CBOW introduced at all when the former method is doing the exact same job?

• CBOW is not equivalent to matrix factorisation of a PMI matrix, it is SkipGram whose loss function is minimised when $$W^\top C=PMI$$ (see Goldberg & Levy paper, 2014)