Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I have a collection of documents and I'd like to extract the most important words and phrases from the entire corpus.
My understanding of TF-IDF is that it is calculated per token per document, so the calculated weights are relative to a given document in the corpus. Is there a way to use TF-IDF to recover the most significant terms in the entire corpus, or is this the wrong approach? If the latter, what would be a more appropriate NLP approach?
Provided limited information & context you have provided, I would suggest you to look for feature selection when each dimension belongs to a word. Feature selection will give you most important words. Most important words in the sense, words deciding the decision surface of the model.
