# NLP Text Summarization - which metrics to use in evaluation?

I'm trying to implement Text Summarization task using different algorithms and libraries. To evaluate which one gave the best result I need some metrics. I have read about the Bleu and Rouge metrics but as I have understand both of them need the human reference summaries as a reference. Is it so that no automatic evaluation can be done in text summarization, i.e human generated summary is needed beforehand? What is the common practice for the tasks like this? How often is the text summarization accuracy measured at all?

• Without reference summaries, you cannot get an automatic metric for assessing the quality of the summary. However there is other measure, such as n-gram novelty, for measuring the abstractiveness of a summary for example. It does not need reference summaries. Jun 1, 2020 at 7:19