I am trying to automatically categorize news articles according to their primary topics, i.e. politics, entertainment, sports, business, technology, health, etc.
There are some labeled datasets out there, but ideally I would like to create my own (for potential commercial usage later on). I am using python, but an answer clear enough with relation to any language would be sufficient.
So, what would be the best way to go about this task?
My current thoughts are:
Determine the most popular keywords for each category, then associate each keyword/keyword set with each respective category, and then use an algorithm to apply a "category" label to a large set of scraped articled based on the predefined keyword sets.
Another option is to scrape article from specific sections of news sites where the categories are already specified, and apply them to each individual constituent article.
After I have the trained dataset, I plan to implement Naive Bayes classification method to automatically categorize future articles.
...
As you can see I have some ideas, but because the web is a large and magical place, I assume someone with previous experience doing something like this may be able to reduce my effort expenditure by guiding me toward the most feasible solution.
