Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
As discussed with Sean in this Meta post, I thought it would be nice to have a question which can help people who were confused like me, to know about the differences between text mining and NLP!
So, what are the differences between nlp and text-mining?
I have included my understanding as an answer. If possible, please explain your answer with a brief example!
I agree with Sean's answer. NLP and text mining are usually used for different goals. Also, there is indeed an overlap and both definitions are vogue.
Other than the difference in goal, there is a difference in methods. Text mining techniques are usually shallow and do not consider the text structure. Usually, text mining will use bag-of-words, n-grams and possibly stemming over that.
In NLP methods usually involve the text structure. You can find there sentence splitting, part-of-speech tagging and parse tree construction. Also, NLP methods provide several techniques to capture context and meaning from text.
Typical text mining method will consider the following sentences to indicate happiness while typical NLP methods detect that they are not
I have had this doubt since a long time. So, this post here helped me figure the differences between the two.
So, this is the difference between text mining and NLP:
Text Mining deals with the text itself, while NLP deals with the underlying/latent metadata.
Answering questions like - frequency counts of words, length of the sentence, presence/absence of certain words etc. is text mining.
NLP on the other hand allows you to answer questions like; - What is the sentiment? - What are the keywords? (using POS tagging & parsers) - What category of content it falls under? - Which are the entities in the sentence? And more
Just another point of view, Dig the topic names a bit deeper.
Text mining - mining of text (just as data mining, and the data is text data). mining is about extracting useful information from the available data. information could be patterns in text or matching structure but the semantics in the text is not considered. The goal is not about making the system understand what does the text conveys, rather about providing information to the user based on a certain step by step process.
Natural language processing - Natural language is what humans use for communication. processing such a data is NLP. The data could be speech or text. Thus, the main goal is towards understanding what is the semantic meaning conveyed in it. Now you know why we care about grammatical part of speeches and the lexical relations among them.
speech recognition systems could be a part of NLP, but it has nothing to do with text mining. And, it seems like NLP is the bigger fish and it uses text-mining, but its actually the other way around. text-mining uses NLP, because it makes sense to mine the data when you understand the data semantically.
