- Classification problem
- Input is three text fields
- Output classes are A, B, A&B (Note: A and B are not always exclusive though usually are, hence the 'A&B' class)
- Sci-Kit Learn is the currently used ML library
- Each text field is put through a HashingVectorizer (Note: ngram_range=(1,2))
- The output is then fed to a LinearSVC (note: all experimentation with SVC and it's various kernels either never finishes or produces worse results)
- F1-score output (rounded) is A: 0.98, B: 0.95, A&B: 0.80
- Some of the data lacks information on which class it is, either due to being low quality, that information being left out deliberately, or one or two of the fields being an empty string
- Those data samples are being classified as A/B/A&B even when they are actually unclassifiable
- Attempts to label the unclassifiable examples and add 'unclassifiable' as a fourth class have failed, in that they've resulted in low scores for all four classes as a result
- Outlier detection does not detect these samples in any useful way, as far as I have been able to determine
- How do I handle the unclassifiable input data?
- How could I filter for it before passing it to the model for classification?
- How could I determine if an output classification should be discarded?
- If the solution involves moving from multi-class to multi-label - why? How does that solve the problem without introducing the uncertainty of effectively swapping one production model for another?