Just landed on a Entity Resolution (Matching) project. I am working in python and I was wondering if there's an off-the-shelf implementation of string harmonization functions, including:
- removal of trailing spaces
- removal of multiple spaces
- removal of special characters (eg. ü)
All this is easily done with string and Regex operations but I do not want to reinvent the wheel and tap into something already ready (and perhaps learn from that, shall I need to move on to something custom later). Are there readily available libs for that? Tried nltk, which suggests the use of regex expression, textblob, and googling around but found nothing really helpful so far - probabily because I did not get to use the right keyword.
As an example, it would be useful to have a function that replaces special characters from foreign languages with morphological equivalents - for example "ß" in German with "ss" ("straße" -> "strasse")
Edit: clarification In my question, I am not really specifying the problem at hand. This is because what I am trying to get to is really a "grammar" that would let me and the other ppl in my team define workflows of harmonization/canonicalisation operations.
I am at the moment Working on this as a chain of functions f: string -> string. Whether some steps are part of it, it will depend on the problem at hand. Workflow will be likely tested empirically (we'll pick the workflow that leads to the best performing algo)