# Why is the idf important in tf-idf when it seems to just re-scale your features?

I am trying to understand why tf-idf is useful. As I understand the formula to work out the tf-idf is:

Can someone explain what is wrong with the reasoning below:

Imagine I have 100 documents that have a text corpus of 10,000 words. As I understand each document will have 10,000 features once transformed using tf-idf. For a single feature we get the tf part by counting the number of occurrences of that word in each document. We then multiply that feature by the idf.

TL;DR My question is given that we multiply the entire feature by a constant (the value of log(N/dfi)), how does that help in making a model better. As I understand multiplying a feature by a constant doesn't help as the model can figure out this new scale.

Edit After applying tf we might get a document-term matrix like the following:

word /
doc num. computer walk smell help warmth
1        1        0    2     1    0
2        0        2    3     1    1
3        0        1    0     0    3
4        1        2    0     1    0
5        0        1    2     2    1


Then after tf-idf we might get the following:

word /
doc num. computer    walk        smell      help        warmth
1        1*log(5/2)  0*log(5/4)  2*log(5/3) 1*log(5/4)  0*log(5/3)
2        0*log(5/2)  2*log(5/4)  3*log(5/3) 1*log(5/4)  1*log(5/3)
3        0*log(5/2)  1*log(5/4)  0*log(5/3) 0*log(5/4)  3*log(5/3)
4        1*log(5/2)  2*log(5/4)  0*log(5/3) 1*log(5/4)  0*log(5/3)
5        0*log(5/2)  1*log(5/4)  2*log(5/3) 2*log(5/4)  1*log(5/3)


tf-idf seems to just scale each feature here by the same value so why does that help certain models like decision trees that can handle different scaling to predict better?

• log(N/dfi)) is not constant at all, It will depend on the specific term, the only constant will be N (the number of documents) So you weight each term according to its pretense across documents. A term that appears in all documents will be consider lees important (log(1) = 0) This is specially evident when you have stop words in your documents (you have not cleaned your corpus) since stop words will appear in almost all documents – Julio Jesus Feb 18 at 16:55
• But it doesn't depend on the document. Every document will get multiplied by the same value for that feature – Daniel Wyatt Feb 18 at 16:57
• But remember that your features on this vectorization will be the words not the documents, you want to give weight to each word(feature) the documents are your observations. I do not know if I'm being clear – Julio Jesus Feb 18 at 17:00
• TL;DR My question is given that we multiply the entire feature by a constant You do not, because your features are the words not the documents – Julio Jesus Feb 18 at 17:02
• Yes I agree with that. But in a normal model (say linear regression) I wouldn't think that changing a feature by multiplying it by a constant is that useful. Why isn't that the case here? – Daniel Wyatt Feb 18 at 17:03