extract names in a list of names

Question

I have been provided with a text cleaning task and I am assuming this involves some amount of natural language processing.

I have a collection of names which does not have any specific pattern and I need to separate individual names from that list.

Below is an example:

John Batista Barack Obama George W. Bush Serafim Batzoglou Vijay Pande Fei Fei Li Oussama Khatib Kwame Nkrumah John Oliver

As you can see there are two things peculiar with the paragraph.

1) space " " is the only delimiter, however, it is unusable as there are names which have middle names.

2) I have french, German, Spanish, Chinese and other ethnic sounding names written in English.

Now the challenge I am facing is to find names from this paragraph. I am very new to NLP and use Python and R for modeling work.

Any help will be appreciated.

Answer 1

I'm not an expert here, so here's my (brute force?) method.

SeatGeek has open-sourced a python library called fuzzywuzzy which is great at text matching. It has a function called token_set_ratio that compares two multi-word strings and scores their distance. It can consider just the intersection of individual words and score only that intersection. eg "Barack Obama Bill Clinton Madeleine Albright" will get a score of 100 with "Bill Clinton", but also with "Bill Barack".

You'll need to build a list of known names, judging by the list they're famous people, which a quick google search shows hundreds of sites that have that ready for you. With that list, you can just loop your way through, computing a score with token_set_ratio. If the famous name beats some threshold score against your string, you append it to a list of captured names.