2
$\begingroup$

I am trying to build a classifier that would classify if a document is a document about sports or not. I have enough samples of sports document to train a classifier on, however I can't imagine how I would sample 'not a sports document' category as there can be anything - book, news article, resume, invoice etc. How would approach this problem?

I already tried training One class SVM classifier with my sample of sport documents, however the accuracy turned out to be awful - around 6%.

I also read about PU learning, do you think this is the way to go? Are there any other options?

Thank you.

$\endgroup$
0
$\begingroup$

Your problem is indeed a typical one-class classification problem, and as far as I know one-class SVM is usually a good option for that.

I think you should investigate what causes the poor performance:

  • Evaluating with accuracy is probably not informative enough, you would need to find out at least whether the errors tend to be mostly false positive or false negatives, thus using precision/recall.
  • You could look at what is happening at the level of features: I would expect some words specific to sports to be assigned a strong weight by SVM for instance. It could also be a problem with the dimensionality being too high, maybe you need to remove stop words or filter out rare words, etc.
|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.