I want to know how are the word weights updated for the embedding layer in Keras and for Word2Vec. Like for the normal model.add(Embedding(..)) and from gensim.models import Word2Vec. Though after using Word2Vec() we put them in the Keras Embedding layer. So I want to know how this is being done mathematically.

I've gone through this post, but I just still want a clear mathematical difference between Word2Vec and normal embedding. I want to know the math, how is gradient descent used for updating the weights of the words, how is backpropagation done etc.

P.S. What I understood is that Word2Vec() converts words into vector space and similar words are represented closer to each other. And in case of Embedding, unique integers are assigned, and represented in vector space. So are the words converted to one-hot vectors, or how are they just being put up as numerical values. I want to get this concept cleared, I haven't been able to find any resources which explain this difference mathematically from scratch.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.