I'm just curious are there some alternative techniques to word 2 vector representation? So words/phrases/sentences are not represented as vectors but have a different form. Thanks.
1
-
1$\begingroup$ Welcome to DataScienceSE. Yes, of course: this is the traditional bag of word representation. $\endgroup$– ErwanCommented Nov 5, 2022 at 20:41
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
1
In your question you talk about vector embeddings or "word 2 vector representation" (word2vec was the first software to train word embeddings). It's important to understand that not all vectors are embeddings:
- Embeddings are short vectors made of real numbers, they were invented around 2010. There are many different types of embeddings, i.e. methods to train the embeddings from a corpus: word2vec, Glove, Elmo, Fasttext, Bert...
- Before this, people were also using vectors representing a "bag of words": one-hot-encoding for a single word, frequency count or TFIDF for a sentence/document. These vectors are long and sparse, i.e. they usually contain a lot of zeros.
These are the most common word representation methods, but there are potentially other alternatives. For example, in Wordnet the words are nodes in a graph and relations between words are represented as edges.
-
$\begingroup$ Thank you so much. I've finally got some clarity about it. $\endgroup$– Dron4KCommented Nov 22, 2022 at 18:07
$\begingroup$
$\endgroup$
3
I researched this question and looks like I realized that there are many word representations.
Dictionary lookup
One-hot encoding
Distributional representation
a. Frequency counts
b. Word2Vec(Skip-Gram and CBOW)
c. GloVe
Elmo
Subword
@Erwan provided more correct answer above
-
$\begingroup$ You are confusing things: 'Dictionary lookup' is not a word representation; word2vec, Glove, Elmo and subwords are different word embeddings representations. The one which is correct is in your list is One-hot encoding, i.e. the good old bag of words. $\endgroup$– ErwanCommented Nov 21, 2022 at 14:38
-
$\begingroup$ @Erwan: I wanted to know how else words can be represented other than as vectors(if at all). You prompted to me that it is bag of words. Then I read about bag of words and figured out that it is still vector representation(e.g. machinelearningmastery.com/gentle-introduction-bag-words-model) So, I thought my question is incorrect. Everything is always a vector. It might be correctly to think about embedding representations. I read this article towardsdatascience.com/… and answered. Sorry for confusing $\endgroup$– Dron4KCommented Nov 21, 2022 at 19:36
-
$\begingroup$ ok, I tried to explain things better in another answer. $\endgroup$– ErwanCommented Nov 21, 2022 at 23:58