Most of the open source datasets are well formatted i.e each email message is separated well like the enron email dataset. But out in the real world it is highly difficult to separate a top email message from a thread of emails.

For example consider the below message.


Can you offer me a better discount.

Customer Relations.

---- On Wed, 10 May 2017 04:05:16 -0700 someone@somewhere.com wrote ------

Hello Mr.X,

Does the below work out. Do let us know your thoughts.


Sales Manager.

Now the reason why we want to split the emails is because we want to do sentiment analysis. When we fail to split the email then the results will be wrong.

I searched around and found this very comprehensive research paper. Also found an implementation by mail gun called mail gun talon. But unfortunately it does not work well for certain kind of patters.

For example when the second message in the email thread breaks like

---------- Forwarded message ---------- 

instead of the above

---- On Wed, 10 May 2017 04:05:16 -0700 someone@somewhere.com wrote ------

My question is many people who are trying to do such stuff would have definitely faced such problems, but yet the area remains pretty shady. Is there any pretty solid implementation of the paper or something else that splits email pretty well.

  • $\begingroup$ is this link not helpful for you??? $\endgroup$ – Abhishek Verma Jun 5 '17 at 7:25
  • $\begingroup$ it describes the problem in a generic way, but I just want to see if there is any ready made utility/library to do this. It is helpful but there is too much to study and implement. $\endgroup$ – Greedy Coder Jun 5 '17 at 7:26
  • 1
    $\begingroup$ Don't know if you've already seen it, but the author of the paper you cited has available in his website an implementation of their methodology in a software called jangada, maybe tweak around with it? $\endgroup$ – Bogas Jun 8 '17 at 11:59
  • $\begingroup$ @Bogas : Thanks. I have not seen that before. Will take a look at how it works. $\endgroup$ – Greedy Coder Jun 9 '17 at 5:16

First you can read about the format of ARPA internet text messages.

You're going to have to one up on Regex parsing if you are going to want to extract any meaningful info from the emails. I'd suggest the Oreilly book on regex, or reading over regular expressions.

There is also a tool to capture text from incoming emails and write to a text file.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.