4
$\begingroup$

I have a list of email subjects like:

<XYZ> commented on <ABC>
Weekly review for <Company>
Your account is ready 

And I want to find patterns in them so I can group them.

Is there a well known algorithm I can use?

  • Preferably with wide language implementations or easy re-implementation.
  • The algorithm should be unsupervised.
  • The number of different emails is not known.

Update:

I think I can break this down into two problems:

  1. Group subjects by the similar words they use, resulting in the following. Each group should be very distinct from the rest (they should be almost perfectly exclusive) and the algorithm should give relatively small number of groups with good length of the common words.

    [commented, on]
    [weekly, review]
    [your, account, is, ready]
    
  2. Once grouped, it should be easy to find a state automaton that accepts only the group's subject and thus eliminates variable

  3. Then I can go back and check if there are any intersections and tweak the variables.

Having said that, is it better to use a completley different approach like neural nets maybe? I have zero experience with those, but if it makes more sense, I am open to learning.

$\endgroup$
1
$\begingroup$

As per your problem and the description, I'll suggest you to try Text Mining - Bag of Words approach. I did something similar using the same approach and it was really helpful.

More details about the approach are here.

Give it a try. Cheers!

| improve this answer | |
$\endgroup$
0
$\begingroup$

You can associate the right semantic to each letter, start with the definition of the semantic. Build your model from scratch, use rigor to moove on forward. At this point you have the semantic of letter but which is the semantic of a word ? Can we reduce it to something more dense ?

I was jocking but not :)

The question is, how can you categorise data ?

You have to put rules/limits to define something. If you want your work works for a mean interpretation, thus use the mean meaning of each letter/word/sentance/pararagraph.

You can make disjonctions, for example, good(0) bad(1) => lucky = 0 , unlucky = 1 But you can say that be lucky for a moment is be unlucky later thus lucky = 1 Thus you have to put hard limits and build on it, you have 2 options, look backward or moove forward !

With the hope to be helpful !

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.