I'm working to build an opt-out filter for my company. I have a small amount of machine learning experience (I've done a few projects with tensorflow in the past), but wanted to get other opinions on the best approach here.
My problem
We have a small dataset of messages (a few hundred) that have been labeled as "opt-out". They are all pretty similar in content, but different enough that we don't just want to do a fuzzy SQL query. The following are examples of messages that we want to label as "opt-out":
- "Not interested. Please stop texts"
- "Pls stop texting"
- "Not real sure how you got my number... but I am not interested"
On the other hand, >99% of our messages are ok and should not be filtered. There are other interesting cases that I'd like the algorithm to pick out, for example:
"Can you stop texting me and email me instead" is actually clearly not an opt-out.
Question
From the reading I've done so far, it sounds like a RNN with LSTMs are the way to go, but I have very little understanding of these (yet) beyond the basic concept.
I'm interested in any thoughts or insights as to how I might proceed in this specific case where the size of negative examples is so much smaller than the set of positive examples. Would this aspect change the way in which is should train or evaluate my data?
