I have a use case where I have text data entered by an approver while approving of some loan.

I have to make some inferences as to what could be the reasons for approval using NLP. How should I go about it?

It's a Non english language. Can Clustering of text help?? Is it possible to cluster TEXT OF non English language using python libraries.

  • $\begingroup$ Unsupervised text learning could start with topic modeling: rstudio-pubs-static.s3.amazonaws.com/… $\endgroup$ – Peter Aug 19 '20 at 19:14
  • $\begingroup$ So you have the text of the evaluation where I assume the approver explains the reason, right? Are you trying to summarize the reason given by the approver? Because that's completely different from clustering. $\endgroup$ – Erwan Aug 19 '20 at 19:33
  • $\begingroup$ @Erwan, yes..I want to summarize $\endgroup$ – sayan_sen Aug 24 '20 at 20:26
  • $\begingroup$ @sayan_sen then you should look at text summarization techniques, there are lots of options. clustering won't give you the same thing at all. $\endgroup$ – Erwan Aug 24 '20 at 20:58

Is it possible to cluster TEXT OF non English language using python libraries?

Sure! classic approaches based on Bag-of-Words are language independent. For modern approaches based on DNNs, mostly pre-trained models, you just need to find a model in your language or train one model from scratch (for this you need lots of text in that language). For example in case of using AWS infrastructure, check Object2Vec algorithm.

Can Clustering of text help?

Can help. For instance for an initial labeling you can cluster data into similar texts and labels each according to overal concept. More sophisticated solution (easily implemented in python) is topic modeling e.g. LDA algorithm.

More sophisticated solution is, again, pre-trained models like S-BERT.

In this direction, I also recommend having an analysis on keywords for algorithms like RAKE or YAKE.

Hope it helps!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.