I am looking for a way to classifiy text automatically by specific topics, i don´t have labeled data. Is this a possible/usual method of achieving this? If not, what would be better?
Topic Modelling with Mallet
I noticed that mallet finds interesting topics, but these topics just match partially with the topic i want to identify. So the idea is to iterate over all interesting k and search for a topic that meets the requirements with the highest percentage.
- Find with elbow several (good coherence score) k topics (=list of good number of k)
- Have a look at terms with a high frequency and (manually) map these terms to desired topics (= word list for each searched topic)
- model topics for all good k and search for a good match with the word list
I would like to classify/identify text that is about the topic "pollution of the environment"
- [17, 15, 20, 33, 41, 42, 55, 120] k => with good coherence score
- [dirt, pollution, emission, contamination] => high frequency, and these words would match with the topic
- found with k=33 a topic which match these keywords.
Topic "pollution of the environment"
[('pollution', 0.09756215849203013), ('dirt', 0.09028743250670891), ('emission', 0.05491609816030263), ('contamination', 0.02589802450774354), ('sea', 0.02088654660674448), ('ocean', 0.017281515729574187), ('climate', 0.014694946490348864), ('crisis', 0.013676484852403893), ('waste', 0.011380904652591419), ('smog', 0.009812797051311068)])