I am working on a classification problem where I would like to separate business emails from personal emails to analyse their behaviours separately. I am thinking about using regex but after looking at the emails, I realised that the email addresses vary significantly. Although I could manually spot some emails with obvious business names, such as xxx@carsales.com.au, it is hard to filter these emails systematically. Are there some useful techniques I could use? Thx

  • 1
    $\begingroup$ Simple regex filtering is where i would start with $\endgroup$ – Aditya Jun 23 at 13:45
  • $\begingroup$ By behaviours what do explicit mean? I agree regex is good heuristic to start with to initially filter e-mails, but it might be a good idea to look at the content itself to classify emails. Once you have clarified the above, I will gladly give you a more detailed response. $\endgroup$ – shepan6 Jun 23 at 19:43
  • $\begingroup$ @shepan6 Basically we want to analyse demographics of donors to help charities to organise their campaigns to target the right audience. It is after looking at the content of the email addresses that I realised the challenge. Because it involves common sense to discriminate company names from person names in the emails and I am not sure how to do this systematically for all emails.There are 200,000 samples, is this even possible? $\endgroup$ – nilsinelabore Jun 24 at 5:01

Thank you for clarifying your question.

So, from what I have got, you want to predict user demographics from the e-mails (i.e. the text). If you knew some of the demographics for a same group of emails, you could do a supervised task by:

  • Using a Recurrent Neural Network (RNN) or LSTM (https://towardsdatascience.com/understanding-rnn-and-lstm-f7cdf6dfc14e), you can encode the emails by feeding each email into the network as sequence of tokens. From this we get a "hidden representation". From this "hidden representation", we then decode this with a standard neural network, which then outputs demographics (whether it be age, socioeconomic status, etc.)

If you don't know demographics, then you could build customer profiles by agglomerative clustering:

Here, you use something like Doc2Vec (https://medium.com/wisio/a-gentle-introduction-to-doc2vec-db3e8c0cce5e) to encode the emails into vectors in semantic space and then use agglomerative clustering (https://www.datanovia.com/en/lessons/agglomerative-hierarchical-clustering/) over these vectors to find clusters of users.

The above is an unsupervised approach and you can also decide after looking at the resulting dendrogram, how many customer profiles you think are necessary.

| improve this answer | |
  • $\begingroup$ By demographics do you mean identity of the donor? We do have the gender, age etc for some cases, but I'd like to know if we could extract some extra information such as if the person represents a company(this is not in the data apart from the emails). $\endgroup$ – nilsinelabore Jun 24 at 10:28
  • $\begingroup$ Yes, that what I mean by demographics. if you have that information, then best to do a supervised task similar to one I suggested above. To classify whether emails belong to a business then you could see if you could label some of the emails (business / non-business) and put this into the supervised classification task as well. $\endgroup$ – shepan6 Jun 24 at 10:33
  • $\begingroup$ I think it'd be a bit time consuming to manually label identities due to tight time frame and large dataset in this project. But thank you for sharing some insight! $\endgroup$ – nilsinelabore Jun 24 at 11:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.