I am working on a classification problem where I would like to separate business emails from personal emails to analyse their behaviours separately. I am thinking about using regex but after looking at the emails, I realised that the email addresses vary significantly. Although I could manually spot some emails with obvious business names, such as firstname.lastname@example.org, it is hard to filter these emails systematically. Are there some useful techniques I could use? Thx
Thank you for clarifying your question.
So, from what I have got, you want to predict user demographics from the e-mails (i.e. the text). If you knew some of the demographics for a same group of emails, you could do a supervised task by:
- Using a Recurrent Neural Network (RNN) or LSTM (https://towardsdatascience.com/understanding-rnn-and-lstm-f7cdf6dfc14e), you can encode the emails by feeding each email into the network as sequence of tokens. From this we get a "hidden representation". From this "hidden representation", we then decode this with a standard neural network, which then outputs demographics (whether it be age, socioeconomic status, etc.)
If you don't know demographics, then you could build customer profiles by agglomerative clustering:
Here, you use something like Doc2Vec (https://medium.com/wisio/a-gentle-introduction-to-doc2vec-db3e8c0cce5e) to encode the emails into vectors in semantic space and then use agglomerative clustering (https://www.datanovia.com/en/lessons/agglomerative-hierarchical-clustering/) over these vectors to find clusters of users.
The above is an unsupervised approach and you can also decide after looking at the resulting dendrogram, how many customer profiles you think are necessary.