I have files like the following: https://pastebin.com/5mkXY1aU
These are created by filling predefined forms so there are thousands of them that would match a pattern.
I will try to give more generic example for the sake of simplicity.
Let's say I have thousands of documents created using different patterns like `
- "sjkghkjfs <data> skjfs <data> kjskdfjsfkjs <data> sahkj"; - "tretyer erytewr fskjdf <data> trjk"; - "sdhfjsdhj <data> <data> <data> dsjadh"; - "<data> djfhsdk";` and so on.
<data> could be just any string.
I don't know what these patterns are beforehand. I want to find them out.
I can easily come up with some bruteforce solution but it would obviously not work on such amount of data.
It sounds to me like a problem that might have already been solved.
I wondered if there is some readily available software tool I could utilize directly to solve this or a library atleast ?
In case there isn't one how should I proceed to solve it in the most painless way ?
It is a one time task to find these patterns out, so I am not into the idea of spending days on implementing a solution from the scratch.