I have a list of email subjects like:
<XYZ> commented on <ABC> Weekly review for <Company> Your account is ready
And I want to find patterns in them so I can group them.
Is there a well known algorithm I can use?
- Preferably with wide language implementations or easy re-implementation.
- The algorithm should be unsupervised.
- The number of different emails is not known.
I think I can break this down into two problems:
Group subjects by the similar words they use, resulting in the following. Each group should be very distinct from the rest (they should be almost perfectly exclusive) and the algorithm should give relatively small number of groups with good length of the common words.
[commented, on] [weekly, review] [your, account, is, ready]
Once grouped, it should be easy to find a state automaton that accepts only the group's subject and thus eliminates variable
Then I can go back and check if there are any intersections and tweak the variables.
Having said that, is it better to use a completley different approach like neural nets maybe? I have zero experience with those, but if it makes more sense, I am open to learning.