Let's say I have a text corpus with inconsistently written bi-grams. An example would be "bi gram", "bi-gram", "bigram". Is there any standard text preprocessing method to normalize all these as the same thing? i.e. replace all such occurrences by "bigram". I should also mention that I have no prior knowledge of what exact bi-grams are present in the corpus.

  • $\begingroup$ What's the reason for the downvote? I'm not sure what's wrong with the question $\endgroup$ – yee Feb 10 at 7:47
  • $\begingroup$ It seems you are asking two questions at the same time. Try to post your second question separately. $\endgroup$ – lcrmorin Feb 10 at 20:18
  • $\begingroup$ @lcrmorin: Thanks! I have edited the question accordingly $\endgroup$ – yee Feb 11 at 6:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.