I have a pipeline built which at the end outputs a bunch (thousands to tens of thousands or more) of named entities. I'd like to do aggregates on those named entities (to see, e.g. how many times a specific named entity is mentioned in my corpus). A problem that I am arriving at; however, is that the named entities often don't match up with each other even though they are the same entity. For example, one instance of the named entity might be "Dr. John Smith" while another instance is "John Smith" or one instance might be "Google" while another might be "Google Inc.". This makes aggregating quite hard to do.
In order to deal with this issue and set "Dr. John Smith" to be the same entity as "John Smith", I was thinking of doing word matching between my named entities. I.e. I would check if named entity A has a word in common with named entity B and if they do set them to be the same entity. This approach is obviously seriously flawed. I will be equating "John Nguyen" and "John Smith" as the same entity even though they are obviously not. What's potentially even worse with this method though is I might run into similarity chains where I have "John Smith" linked with "Richard Smith" linked with "Richard Sporting Goods Inc." linked with "Google Inc." etc etc... While I may be willing to allow issues arising from former problem through, the latter problem appears to be catastrophic.
Are there any accepted techniques in the NLP community for dealing with this issue?