2
$\begingroup$

Word2vec looks excellent to me as representation of corpus for sentiment analysis. It has relations between words etc. TF-IDF has only weight of the word how important it is. Results with sentiment analysis using both of these representation are quite similar ~90%

Why TF-IDF has such a good results?

$\endgroup$
3
  • 1
    $\begingroup$ Do you have any references for your comparison/benchmark? Which corpus? $\endgroup$ – n1k31t4 Jun 13 '18 at 19:52
  • $\begingroup$ Typical examples with imdb dataset $\endgroup$ – wpazio Jun 13 '18 at 20:07
  • $\begingroup$ I've mentioned it in the opening of my answer below, but how the sentiments are related to the word embeddings will play a big part. I will update/delete my answer if it doesn't help you. $\endgroup$ – n1k31t4 Jun 13 '18 at 20:19
1
$\begingroup$

One important factor to take into account is how you use the numerical representation of words / embeddings from either TF-IDF or Word2Vec to then compute sentiments. Without knowing how you do this, it is difficult to give a concrete answer. Also, which task are you working on, what does a result of 90% mean?

Regardless of how you compute TF-IDF (there are several definitions - shown below), it is essentially assigning a numerical value to a word, thus creating a mappng of sorts. Word2Vec technically created an embedding, as it maps individual words into a vector space.

enter image description here

enter image description here

Images taken from here

The final TF_IDF is simply the multiplication of the term frequency and the inverse document frequency.

I won't go into details as to how the vectors in Word2Vec are computed, but they also define a way to assign a numerical vector (an embedding) to a single word. In essence, both of these are saying how important a word is, in the context of your documents (your corpus), with Word2Vec also having the interpretability of comparison between word vectors. For example, doing this with the associated vectors of the words actually works really well:

King - Man + Woman = Queen

Perhaps the sentiments you are computing, based on either of these methods , does something similar to taking an average over many words appearing in a sentence, and you end up with a normalised and similar results.


TF-IDF takes a more intuitive approach, looking at how many times a word appears in general, in how many of the documents does it appear and how many times. Word2Vec instead looks at which words often appear together (there is a related quote that normally gets brought up here: "You can judge a person by the company they keep"). So the intuition behind each on is slightly different, but you have numerical values for each word. Perhaps a closer comparison of TF-IDF would be to look at Doc2Vec.

There is also the GLoVE embedding model, which scores well in many NLP tasks - on the same level as Word2Vec embeddings.

$\endgroup$
3
  • $\begingroup$ Just one question, Is tfidf also a word-embedding ? Can word2vec and tfidf be compared e.g. as a different word embedding methods? $\endgroup$ – wpazio Jun 13 '18 at 23:28
  • 1
    $\begingroup$ @robinloop tfidf is not usually considered a word embedding, because it maps each word to a single scalar value, whilst an embedding should map a word into a vector space. You might consider the combined bag-of-words vector for a document using tfidf to be an embedding of sorts, albeit with different way of considering context than e.g. word2vec. $\endgroup$ – Neil Slater Jun 14 '18 at 7:08
  • $\begingroup$ Thanks @NeilSlater - I cleared up my definitions a bit to reflect the important distinction you explained. $\endgroup$ – n1k31t4 Jun 14 '18 at 11:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.