Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I am trying to make a text summarization program that will take a text article and reduce it to a para or 2.
Since I am a newbie with no idea of NLP, it is hard to approach and break down the problem. So I was wondering if there was a measure that is used to check for effectiveness and correctness of text summarization. I tried googling this, but nothing that suits my purpose.
Does something like this even exist? or am I going in the wrong direction?
You will first need a human written summary as your correct output. You can then compare the original and generated ones using Rouge scores. They compare the similarity between two giving paras by comparing different combinations of sub-phrases. Search for Rouge and you'll get different types of rouge scores, you'll probably need to use more than one. Also, there are libraries available that calculate rouge scores so you don't have to implement it yourself!
