I want to tackle email similarity with a word embedding approach, not an expert in embedding text but it is possible to embed emails where similar emails have similar vectors? is that okay?. I know that I can do text similarity approaches like Jacard or Levenhenstein but that would be too expensive for the job that I want to do because I will need to compare email from user i against all other users emails and do this same thing for all user ( i try already this and is too expensive ). I thought maybe i was able to embed the emails somehow ( this is my question - which way would be the best way to do that ? ). My final goal would say that vector from the embed of [email protected] would be very similar to [email protected]. There is any approach which could help me with the creation of this embedding?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
There's a bit of confusion about your task: first, apparently by "emails" you mean email addresses, not the full content of the emails, right?
What you're doing looks like record linkage. The complexity of comparing every pair of records (here email addresses) can be decreased using the technique called blocking, where roughly similar pairs are grouped together in order to minimize the number of real comparisons (see Wikipedia page).
Now you're talking about using embeddings, and in this case what you'd need is character embeddings. However it's very unlikely that embeddings will help with the efficiency issue, quite the opposite actually.
