I am trying to identify a particular intent in a corpus of text messages. The data is very imbalanced (maybe 1000 true labels in a 200 000 text messages).
Out of those intents, I also need to make sure this is the only intent in the message (the rest should be only informative).
My first idea was to find a regex that could help me reduce a lot the data but from what I found I still keep a lot of false labels so too imbalanced and it would be too long to label manually.
Another option I found is to use sentence embedding: I aggregate the individual word vectors of each of my sentences in each text message, then I find sentences similar to the true intent I'm trying to find (through cosine similarity). I get back to the original message from which the sentence is taken from and I add this sentence to my data to label.
Then I train a deep learning model with the labeled data.
And at prediction time, it would require to calculate the similarity of each sentence in the message, then if it's superior to a particular threshold, I use the model to predict if the message is of that intent or not.
I was also thinking to use RASA NLU with intent classifier.
Have you got any idea on a good way to proceed ? Thanks