1
$\begingroup$

I have a list of E-mails, and I'm trying to find "similar" E-mails.

For example:

DanB@gmail.com

DanB1@gmail.com

Dan12B@yahoo.com

will be "similar" for me.

I'll be happy to hear if you are fimiliar with a method which can be applied here.

P.S. my available tools are Teradata (SQL) or R.

Thank you in advance

$\endgroup$
2
  • $\begingroup$ Is this question about email addresses? $\endgroup$
    – aventurin
    Sep 21 '16 at 19:46
  • $\begingroup$ Yes.. sorry if it wasnt clear $\endgroup$
    – sagivmal
    Sep 21 '16 at 19:47
3
$\begingroup$

Since your question is super-huge and no one is going to do this job for you, here's a starting point for you to at least let you see how the various string comparison algorithms work:

library(stringdist)
library(stringi)
library(purrr)

emails <- c("ed@microsoft.com", "DanB@gmail.com", "DanB1@gmail.com", "Dan12B@yahoo.com",
            "fred@outlook.com", "bill@juno.com", "Danb21@aol.com", "ed@tumblr.com")

naive_regex <- stri_match_all_regex(emails, "([[:print:]]+)@(.*)")

addressees <- map_chr(naive_regex, 2)

methods <- c("osa", "lv", "dl", "hamming", "lcs", "qgram", "cosine", "jaccard", "jw", "soundex")

map(seq_along(addressees), function(i) {
  map_df(methods, function(method) {
    c(addressees[i] ,method, stringdist(addressees[i], addressees, method=method)) %>% 
      as.list() %>% 
      setNames(c("looking_for", "method", addressees))
  })
})

The output is too long to put here but it's a list of comparison matrices like this:

## [[2]]
## # A tibble: 10 × 9
##    looking_for  method    ed  DanB              DanB1            Dan12B  fred  bill            Danb21
##          <chr>   <chr> <chr> <chr>              <chr>             <chr> <chr> <chr>             <chr>
## 1         DanB     osa     4     0                  1                 2     4     4                 3
## 2         DanB      lv     4     0                  1                 2     4     4                 3
## 3         DanB      dl     4     0                  1                 2     4     4                 3
## 4         DanB hamming   Inf     0                Inf               Inf     4     4               Inf
## 5         DanB     lcs     6     0                  1                 2     8     8                 4
## 6         DanB   qgram     6     0                  1                 2     8     8                 4
## 7         DanB  cosine     1     0  0.105572809000084 0.183503419072274     1     1 0.387627564304205
## 8         DanB jaccard     1     0                0.2 0.333333333333333     1     1 0.571428571428571
## 9         DanB      jw     1     0 0.0666666666666668 0.111111111111111     1     1              0.25
## 10        DanB soundex     1     0                  0                 0     1     1                 0

Once you have an idea of acceptable tolerances, you can then create an algorithm to match => remove / match => remove, etc.

$\endgroup$
1
  • $\begingroup$ I can't vote yet.. but your answer was very helpful.. thank you very much $\endgroup$
    – sagivmal
    Sep 22 '16 at 5:25
1
$\begingroup$

You could try a very simple algorithm: https://en.wikipedia.org/wiki/Levenshtein_distance

https://cran.r-project.org/web/packages/stringdist/stringdist.pdf

After splitting the string by the '@'

$\endgroup$
1
  • $\begingroup$ I can't vote yet.. but your answer was very helpful.. thank you very much $\endgroup$
    – sagivmal
    Sep 22 '16 at 5:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.