3
$\begingroup$

I hope this isn't too basic of a question, I'm banking on the Data Science site description being true where it says "...and those interested in learning more about the field". I'm not looking for programming help, just validation that machine learning could help me with a problem.

I'm trying to find all customer phone numbers in our databases. One database has a field with free-form comments from our customer service center. Here is an obfuscated snippet:

multiple #'s 123-456-7890 and 2345678901...current account #2233445566

As you can see, this record contains two phone numbers and also a 10 digit account number. One of the phone numbers has dashes while the 2nd doesn't. Looking for parenthesis helps, but only finds a small set. There are also other 10 digit numbers that could look like a phone number but in fact aren't.

If I run a query to return all records with a 10 digit number formatted with dashes, I get thousands of records. If I check for any 10 digit number, I get tens of thousands. So manually scanning these records to validate accurate matches is not practical.

I'm wondering if I could build a machine learning model that I could train to accurately find phone numbers in this mess. When I say "accurately", I don't mean 100%, just better than standard SQL queries. If I can, I would use this going forward to parse new data that is created in this database.

It seems to me this problem could be a good candidate for machine learning. But I'm new to machine learning, and the research I've done so far talks about different scenarios that don't seem quite the same.

$\endgroup$
1
  • 1
    $\begingroup$ My impression is that it will be very difficult to apply ML. You would need to find patterns (I suspect you don't have anything obvious, like "#" in the example in real data) in your data, construct features based on these patterns and manually label training data, and even after all this work it does not guarantee you anything. $\endgroup$
    – Akavall
    Nov 1, 2019 at 22:13

3 Answers 3

3
$\begingroup$

In principle this seems close to a NER task, you could try to annotate a sample and train a sequence labeling model on it. However this would require quite a lot of work to get it right: annotation, then probably a good bit of trial and error to tune the right combination of features.

In a case like this I would rather go for a few carefully chosen regular expressions, they are likely to perform at about the same level without requiring as much work.

$\endgroup$
1
  • $\begingroup$ Thanks, @Erwan. I have some good regex already, and it performs pretty well. I'll just stick with that then. :) $\endgroup$ Nov 2, 2019 at 3:41
2
$\begingroup$

I think regular expression is an easier and faster solution than using Machine Learning.

Please check out SO.

https://stackoverflow.com/questions/3868753/find-phone-numbers-in-python-script

$\endgroup$
1
  • $\begingroup$ You're right! I just updated my answer. $\endgroup$
    – ASH
    Feb 28, 2020 at 0:35
0
$\begingroup$

This will parse out the phone number.

import re
Source = """I'm trying to find all customer phone numbers in our databases. One database has a field with free-form comments from our customer service center. Here is an obfuscated snippet:
multiple #'s 123-456-7890 and 2345678901...current account #2233445566
As you can see, this record contains two phone numbers and also a 10 digit account number. One of the phone numbers has dashes while the 2nd doesn't. Looking for parenthesis helps, but only finds a small set. There are also other 10 digit numbers that could look like a phone number but in fact aren't."""

pattern = re.compile(r'(\d{3}[-\.\s]??\d{3}[-\.\s]??\d{4}|\(\d{3}\)\s*\d{3}[-\.\s]??\d{4}|\d{3}[-\.\s]??\d{4})')
for m in re.finditer(pattern, Source):
    print(m)

Result:

<re.Match object; span=(191, 203), match='123-456-7890'>
<re.Match object; span=(208, 218), match='2345678901'>
<re.Match object; span=(238, 248), match='2233445566'>
$\endgroup$
1
  • $\begingroup$ This will only catch numbers of the form ###-###-####. The questioner gives an example of a phone number without dashes. In any case, if you wanted to match just that pattern, a single regex will do it (and much faster). $\endgroup$
    – A_P
    Feb 27, 2020 at 23:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.