I am working with a text classification system. Here, my data-set has around 30 intents. But the problem is I have no system developed to handle inputs that don't go under any of the intents. So, in this case, I am going to train a One class classification model. To teach it how normal data looks, I am going to merge all my training data (of all intents) and train with them. Which feature-extraction and algorithm combo will be best suited for these kinds of tasks? Is there any trade-off, best practice, or other information to know?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Just have n_classes + 1 intents. Name the last one as 'Others'. Accumulate some data that doesn't fit into any of your 30 intents in this class. Treat it as new intent and train your model for these 31 intents/classes. The best SOTA model for text classification would be microsoft/deberta-v3-base. You should be good to go. And evaluate appropriately.
