Building out a system that tries to apply zero or more predefined labels to text.
For each label, we've:
- built out a reasonably good vocabulary of high-value words/features
- developed a corpus containing thousands of labeled entries
- trained a NaiveBayesClassifier for each topic that does a good job of classifying valid vs noisy content
The problem seems to be that the individual classifier is great at differentiating between valid & noisy content WITHIN a topic:
- "the green energy bill will revolutionize..." (green = "green energy")
- "the green bay packers went on to lose their..." (green != "green energy")
...but when classifying content that shouldn't match ANY topic it has a very high rate of false positives. There's no "everything else" label!
tl;dr it's good at subtle, in-topic differentiation, but terrible at broad topic labeling
Are there any algorithms that help you classify into N categories, but allow for "everything else" which might not fit into ANY of the categories?