1
$\begingroup$

I have a bunch of articles about science from a certain website. When a new article is published, I want to determine if that article is really talking about science (and not politics for example). How can I do that? What machine learning technique shroud I use? I'm thinking at using something similar to spam detection. Is that ok? Thank you!

$\endgroup$
0
$\begingroup$

Spam detection can be done with many different methods, the same goes for your task. They do share the similar idea of processing a given text and classifying it to be one of 2 classes (science/not-science or spam/not-spam).

What you first need to do is to turn the articles into a vector of constant size (for example with Word2vec which takes as its input a text and produces a vector space).

Ones you have a vector representing each article, you can start training your classifier and feature extractor (these days they are trained together).

As for determining which machine learning approach to take, you can try first using an SVM, it will probably be good enough.

You can follow one of the following tutorials (there are many more), just replace their dataset with yours :

Email Spam Filtering: An Implementation with Python and Scikit-learn

Spam Classifier in Python from scratch

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.