What are the exact differences between Word Embedding and Word Vectorization?

Question

I am learning NLP. I have tried to figure out the exact difference between Word Embedding and Word Vectorization. However, seems like some articles use these words interchangeably. But I think there must be some sort of differences.

In Vectorization, I came across these vectorizers:

CountVectorizer, HashingVectorizer, TFIDFVectorizer

Moreover, while I was trying to understand the word embedding. I found these tools.

Bag of words, Word2Vec

Would you please briefly summarize the differences and the algorithms of between Word Embeddings and Word Vectorization? Thanks a lot.

Answer 1

A „bag of words“ usually describes encoding of text where one word (or ngram) is represented as one variable (column). This can be done as binary encoding or as count of words, often called one-hot encoding. Alternatively, you can introduce weights to represent the frequency of words in a document, such as TFIDF. See also here for a sklearn implementation. Hashing essentially is a „bag of words“ using the hashing trick to cope with previously unseen words in a corpus and a large (or growing) corpus.

In word2vec, each word is represented by a vector, which indicates how close one word is to another (this is the result of a pre-trained model). You can use a pre-trained word2vec model and assess the proximity of words by comparing two (word) vectors e.g. based on the Euclidean distance. These vectors help models to better understand the (semantic) structure of some text via understanding the empirical co-occurance of words (which is not possible with one-hot encoding etc.)

BERT goes even one step further. In BERT pre-training a word in a sentence is „masked“, where the model tries to predict the masked word in a sentence. Also „next sentence prediction“ is used to pre-train BERT models. By doing so, BERT has a even better ability to understand semantic relations in a text.

Answer 2

I believe "embedding" is simply a subtype of "vectorization" where you use neural networks to learn the vectorization.

As stated by Peter above, you can vectorize a text without deep learning, but I don't think I've seen the word 'embedding' used in a non-deep learning context.

So vectorization refers to the general process of converting text or characters to a vector representation while embedding refers to learning the vectorization through deep learning (often through an embedding layer).