I have a data set of customers with complaints.

As a result, I want to perform topic modelling to find topics that customers talk about in their complaints. I use LDA for this. In the results of LDA, I have seen that there are semantically similar words in the topics. I think it will help LDA to group all semantically very similar words and replace them with a representative.

I would build as a dictionary { representative: [ list of words] }

In preprocessing, I would then replace all words within the complaint texts with their representative.

This would mean that the complaint texts would contain more common words and LDA would probably be able to find better topics.

Is this idea correct?

How can I find such a semantic grouping within all words to find representatives and their corresponding word list?

I am currently using the spacy framework.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.