In NLP word embedding represent word as number but after reading many blog i found that word are represent as vectors ? so what is word embedding exactly and Why words are represented in vector and the vector size is huge. what values dose that vector represent ?

what is the difference between word and character embedding ? please explain in simple term with simple example.


Most problems in NLP require the system to understand the semantic meaning of the text and not just the arrangement of specific words.

Semantic understanding enables a system to say that, "I am happy" and "It's joyful", have the same meaning.

To incorporate this feature to a system, we present words of a particular language in form of vectors. Often called as embeddings, they help in establishing similarities between words and phrases.

For instance, a vector representing the word "happy" will lie in the vicinity of the vectors representing the words "joy", "pleasure", "sad" etc. These vectors are high dimensional but using PCA or other dimensionality reduction techniques they are brought down to 3 dimensions where they could be visualized.

That's why we encode words in the form of vectors. We often use cosine similarity to determine the closest vector to a given vector in analysing sematic similarity.

For an intuition, the 3D space which contains vectors for all possible English words could be thought of our knowledge base. We tend to keep similar words together in our mind. If we are talking about fast food, for instance, our brain would capture the region of the knowledge base to retreive words related with fast food like "burgers", "chicken" etc.

  • $\begingroup$ Thank you @Shubham Panchal .Say for example the word "happy" is represented as vector with 3dimensions/values[0.01,0.02,-0.01] Assume for the sake of my understanding 0.01 represent distance between happy and joy. 0.02 represent distance between happy and pleasure and -0.01 represent distance between happy and sad (keeping minus as its opposite ) . Similar the vectors for joy,pleasure,sad have same values .Is my understanding correct ? Is there any ways to visualize the actual words represented by values in a particular word vector values like how i have shown it . $\endgroup$ – Aj_MLstater Oct 9 at 14:42
  • $\begingroup$ Yes, exactly. We may consider the euclidean distance between them $\endgroup$ – Shubham Panchal Oct 10 at 0:29
  • $\begingroup$ thank you could you please also explain in simple terms about character embedding as well ? $\endgroup$ – Aj_MLstater Oct 10 at 5:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.