5
$\begingroup$

I found a data set called Enron Email Dataset. It is possibly the only substantial collection of "real" email that is public. I found some prior analysis of this work:

  • A paper describing the Enron data was presented at the 2004 CEAS conference.

  • Some experiments associated with this data are described on Ron Bekkerman's home page

  • Parakweet has released an open source set of Enron sentence data, labeled for speech acts.

  • Work at the University of Pennsylvania includes a query dataset for email search as well as a tool for generating spelling errors based on the Enron corpus.

I'm looking for some interesting current trend topics to work with.please give me some suggestions.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ You should take a look at the emails before coming up with any decision. $\endgroup$
    – SmallChess
    Commented May 11, 2015 at 15:43
  • $\begingroup$ ya you right .. i gone through some of them ... most of them are business related topics and some are contentious like a threaded . this data set contain everything .. i mean inbox,deleted,sent items and so on. $\endgroup$
    – Miller
    Commented May 12, 2015 at 10:03
  • 2
    $\begingroup$ If you write a paper using the ideas collected here, will you list StackOverflow as one of your collaborators? :) $\endgroup$
    – logc
    Commented May 26, 2015 at 9:43
  • $\begingroup$ why not... sure.. :P $\endgroup$
    – Miller
    Commented Dec 22, 2015 at 9:13

5 Answers 5

Reset to default
3
$\begingroup$

You're learning, are you? Try to find something easy and interesting to start. Why don't you start off something easy like building a Bayesian model to predict which email will get deleted. You should glance over those deleted emails, are they spams? are they just garbage?

Here, you have a simply supervised model where the data-set already labels the emails for you (deleted or not). Think of something easy like words, titles, length of the email etc, see if you can build a model that predicts email deletion.

Improve this answer
$\endgroup$
2
  • $\begingroup$ thanks ... i got some interesting thoughts from your comment . specially titles .. because those are the first thing that decide whether i would like to open it or not . i thought something with the email marketing. what sort of email got more attention, kind of analysis . and also your statistical analysis things also good . $\endgroup$
    – Miller
    Commented May 12, 2015 at 14:06
  • $\begingroup$ I suggest you should go for it. You aren't doing a Google email filter, try something easy and simple. $\endgroup$
    – SmallChess
    Commented May 12, 2015 at 23:39
4
$\begingroup$

The following are some research that can done on e-mail dataset:

  • linguistic analysis to abbreviate an email message

  • Categorize e-mail as spam/ham using machine learning techniques.

  • identifying concepts expressed in a collection of email messages, and organizing them into an ontology or taxonomy for browsing

Improve this answer
$\endgroup$
3
$\begingroup$

Wonderful dataset with many opportunities to brush up on text analysis skills!

My first thought would be to try some Topic Modelling on the dataset. If you are using Python there is a library I've used called gensim which has some fairly thorough tutorials to get you started. A friend of mine did something similar with the Enron dataset, using parallelized preprocessing and distributed latent Dirichlet allocation to infer topics over the email corpus.

Improve this answer
$\endgroup$
1
$\begingroup$

The first idea in my mind is : view it forming a social graph where nodes are the email-ids (people) and 2 nodes are connected if they communicate with each other. You can also view it as a weighted graph where weight comes from the frequency of conversation and you can add the sense of direction as well using the sender-receiver information. Now you can apply all sort of social network analysis on this.

Improve this answer
$\endgroup$
0
$\begingroup$

One thing I cannot see here is:

  • Cleaning, pre-processing and structuring

Although this does not sound too exciting, it is probably the most important thing when it comes to working with "real" data. Real data is never clean. If you receive an Email data dump you'll find all kinds of garbage. Forwarded messages, different kinds of quotation styles, different languages (or mixes), bullet point lists etc.

Sometimes you'll see messages with 99% garbage and only one line with actual information embedded in a stream of forwarded messages etc.

Learning how to clean this data and extracting the relevant information is actually the hard part. It's not that fancy, everybody likes to work with clean data, but it's nevertheless another thing that can be done with an Email data set that comes from the real world.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.