5
$\begingroup$

I am wondering if there are any public datasets of Google news with various news categories such as politics, entertainment, lifestyle, general news, sports etc.

I want to use such dataset for topic detection of various sentences or paragraphs. I was planning to train a classifier with such a dataset and use it for predictions. However, I couldn't find any. Are there any such known datasets available?

$\endgroup$
5
$\begingroup$

This dataset is included with scikit-learn, a popular ML library for Python.

http://scikit-learn.org/stable/datasets/twenty_newsgroups.html

It is postings to Usenet and categorized by the group. The group titles are not exactly "categories" like you would see on Google News, but each newsgroup is supposed to be on a specific topic as indicated by the name, so the concepts are similar. For example:

  • alt.atheism, - Atheism
  • comp.graphics, - Computer Graphics
  • ...
  • rec.autos - Automobiles
  • rec.motorcycles - Motorcycles
$\endgroup$
  • 1
    $\begingroup$ it is a relevant dataset but very small with limited categories. $\endgroup$ – utengr Sep 28 '17 at 10:23
4
$\begingroup$

Here is a massive dataset of news with categories which I created for exactly such a reason.

https://www.kaggle.com/therohk/india-headlines-news-dataset

Includes ALL headlines published by Times of India from 2001-2017 with categories.

Contains 2.3 million entries.

$\endgroup$
  • 1
    $\begingroup$ It's an interesting dataset, however, too specific for my use-case. It probably won't generalize well to other news websites. Moreover, headlines do not have enough words for topic modeling. I have tested another such dataset with million headlines and it performed poorly. $\endgroup$ – utengr Feb 14 '18 at 12:21
  • $\begingroup$ Thanks for the feedback. The other dataset you tried is probably mine too. I'm soon putting up a wider dataset with mixed sources from all over the world. This should give better generalisation. However, wont help much if the length of text is an issue. $\endgroup$ – Rohit Feb 15 '18 at 12:50
0
$\begingroup$

There is another big news dataset in Kaggle called All The News you can dwnload it Here.

The data primarily falls between the years of 2016 and July 2017. And were scraped with beautiful soup from big US news sites like: New York Times, Breitbart, CNN, Business Insider, the Atlantic, Fox News, Talking Points Memo, Buzzfeed News and many more.

$\endgroup$
  • $\begingroup$ I couldn't find the news categories (classes) for this dataset. $\endgroup$ – cemsazara Aug 21 '18 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.