As per my knowledge, Word2Vec is belongs to non-contextual embedding technique. this have only semantic relationship between words.

We can implement Word2Vec, either in CBoW or skip-gram model. but i confused with below statements:

  1. The CBOW model is designed to predict a target word based on its surrounding context words.
  2. Unlike the Skip-gram model, which predicts context words given a target word, CBOW focuses on predicting the target word itself.

since word2vec is non-contextual. but in CBoW, it is considering the context to find the target. can you please give more insights about these two(CBoW, skip-gram).

  • $\begingroup$ The difference between contextual and non-contextual embeddings is to use or not the context at inference time. Neither CBoW nor skipgram word embeddings use the context at inference time, while BERT and RoBERTa do. $\endgroup$
    – noe
    Mar 26 at 7:02
  • $\begingroup$ can you please give more details $\endgroup$
    – Tovlk
    Mar 26 at 7:05
  • 1
    $\begingroup$ Please describe specific doubts about what I said. For full info about those methods, there are dozens of online tutorials about word2vec (e.g. original article, tf tutorial) and also about BERT (e.g. original article, illustrated BERT). $\endgroup$
    – noe
    Mar 26 at 7:28


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.