I need a set of news headlines and articles to help me in a project on automatic summarization. Is there such a dataset or something similar?


closed as off-topic by oW_, Stephen Rauch, tuomastik, Djib2011, Siong Thye Goh Sep 12 '18 at 1:31

  • This question does not appear to be about data science, within the scope defined in the help center.
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 4
    $\begingroup$ This is better suited for opendata.stackexchange.com $\endgroup$ – oW_ Sep 10 '18 at 16:57
  • $\begingroup$ Take a look at here. $\endgroup$ – Media Sep 10 '18 at 17:06
  • 1
    $\begingroup$ There's one with Medium.com Blog posts! $\endgroup$ – Aditya Sep 11 '18 at 6:31

The most widely used ones in text summarization research is the DUC dataset. If you see a paper using dataset "DUC 2015" or "DUC 2016" that's from here.

I have also personally used the Reuters arcihve. You just need to download each article with wget or something similar. See also here.

The CNN / DailyMail dataset is also widely used in summarization especially in recent years, although it labels itself as a Q&A dataset.

  • $\begingroup$ How can you log in to get the DUC dataset? I can't figure out how I can register with my institution's email. $\endgroup$ – mosafattah Sep 12 '18 at 9:55
  • $\begingroup$ The link in the answer has instructions about how to request access to the dataset. Why not read it? $\endgroup$ – user12075 Sep 12 '18 at 14:26

Not the answer you're looking for? Browse other questions tagged or ask your own question.