10
$\begingroup$

I am wondering if there are any public datasets of Google news with various news categories such as politics, entertainment, lifestyle, general news, sports etc.

I want to use such dataset for topic detection of various sentences or paragraphs. I was planning to train a classifier with such a dataset and use it for predictions. However, I couldn't find any. Are there any such known datasets available?

Improve this question
$\endgroup$

3 Answers 3

Reset to default
6
$\begingroup$

Here is a massive dataset of news with categories which I created for exactly such a reason.

Includes all the headlines published by Times of India from 2001-2019 with categories.

Contains ~3 million entries.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ It's an interesting dataset, however, too specific for my use-case. It probably won't generalize well to other news websites. Moreover, headlines do not have enough words for topic modeling. I have tested another such dataset with million headlines and it performed poorly. $\endgroup$
    – utengr
    Feb 14, 2018 at 12:21
  • $\begingroup$ @Rohit how did you cluster your articles and assign the categories for each article, what was the process that you used for that? $\endgroup$
    – Amany
    Aug 5, 2021 at 20:18
  • 1
    $\begingroup$ these are human tagged categories, presumably added by the journalist or media manager at the time of publication. i have removed very rare categories by moving them up one level to the parent category. the list of high level categories can been seen under the website's sitemap page. $\endgroup$
    – Rohit
    Aug 7, 2021 at 16:01
5
$\begingroup$

This dataset is included with scikit-learn, a popular ML library for Python.

It is postings to Usenet and categorized by the group. The group titles are not exactly "categories" like you would see on Google News, but each newsgroup is supposed to be on a specific topic as indicated by the name, so the concepts are similar. For example:

  • alt.atheism, - Atheism
  • comp.graphics, - Computer Graphics
  • ...
  • rec.autos - Automobiles
  • rec.motorcycles - Motorcycles
Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ it is a relevant dataset but very small with limited categories. $\endgroup$
    – utengr
    Sep 28, 2017 at 10:23
0
$\begingroup$

There is another big news dataset in Kaggle called All The News you can dwnload it Here.

The data primarily falls between the years of 2016 and July 2017. And were scraped with beautiful soup from big US news sites like: New York Times, Breitbart, CNN, Business Insider, the Atlantic, Fox News, Talking Points Memo, Buzzfeed News and many more.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I couldn't find the news categories (classes) for this dataset. $\endgroup$
    – cemsazara
    Aug 21, 2018 at 17:33

Not the answer you're looking for? Browse other questions tagged or ask your own question.