1
$\begingroup$

I have a list of E-mails, and I'm trying to find "similar" E-mails.

For example:

DanB@gmail.com

DanB1@gmail.com

Dan12B@yahoo.com

will be "similar" for me.

I'll be happy to hear if you are fimiliar with a method which can be applied here.

P.S. my available tools are Teradata (SQL) or R.

Thank you in advance

$\endgroup$
2
  • $\begingroup$ Is this question about email addresses? $\endgroup$
    – aventurin
    Sep 21 '16 at 19:46
  • $\begingroup$ Yes.. sorry if it wasnt clear $\endgroup$
    – sagivmal
    Sep 21 '16 at 19:47
3
$\begingroup$

Since your question is super-huge and no one is going to do this job for you, here's a starting point for you to at least let you see how the various string comparison algorithms work:

library(stringdist)
library(stringi)
library(purrr)

emails <- c("ed@microsoft.com", "DanB@gmail.com", "DanB1@gmail.com", "Dan12B@yahoo.com",
            "fred@outlook.com", "bill@juno.com", "Danb21@aol.com", "ed@tumblr.com")

naive_regex <- stri_match_all_regex(emails, "([[:print:]]+)@(.*)")

addressees <- map_chr(naive_regex, 2)

methods <- c("osa", "lv", "dl", "hamming", "lcs", "qgram", "cosine", "jaccard", "jw", "soundex")

map(seq_along(addressees), function(i) {
  map_df(methods, function(method) {
    c(addressees[i] ,method, stringdist(addressees[i], addressees, method=method)) %>% 
      as.list() %>% 
      setNames(c("looking_for", "method", addressees))
  })
})

The output is too long to put here but it's a list of comparison matrices like this:

## [[2]]
## # A tibble: 10 × 9
##    looking_for  method    ed  DanB              DanB1            Dan12B  fred  bill            Danb21
##          <chr>   <chr> <chr> <chr>              <chr>             <chr> <chr> <chr>             <chr>
## 1         DanB     osa     4     0                  1                 2     4     4                 3
## 2         DanB      lv     4     0                  1                 2     4     4                 3
## 3         DanB      dl     4     0                  1                 2     4     4                 3
## 4         DanB hamming   Inf     0                Inf               Inf     4     4               Inf
## 5         DanB     lcs     6     0                  1                 2     8     8                 4
## 6         DanB   qgram     6     0                  1                 2     8     8                 4
## 7         DanB  cosine     1     0  0.105572809000084 0.183503419072274     1     1 0.387627564304205
## 8         DanB jaccard     1     0                0.2 0.333333333333333     1     1 0.571428571428571
## 9         DanB      jw     1     0 0.0666666666666668 0.111111111111111     1     1              0.25
## 10        DanB soundex     1     0                  0                 0     1     1                 0

Once you have an idea of acceptable tolerances, you can then create an algorithm to match => remove / match => remove, etc.

$\endgroup$
1
  • $\begingroup$ I can't vote yet.. but your answer was very helpful.. thank you very much $\endgroup$
    – sagivmal
    Sep 22 '16 at 5:25
1
$\begingroup$

You could try a very simple algorithm: https://en.wikipedia.org/wiki/Levenshtein_distance

https://cran.r-project.org/web/packages/stringdist/stringdist.pdf

After splitting the string by the '@'

$\endgroup$
1
  • $\begingroup$ I can't vote yet.. but your answer was very helpful.. thank you very much $\endgroup$
    – sagivmal
    Sep 22 '16 at 5:26

This site is temporarily in read only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .