I have many tweets that I would like classify based on their similarity. Unfortunately I am not quite familiar with text-classification and nlp, so I had to read a lot of documents before having an idea on the topic. My tweets have no labels so I cannot classify them: only manually, but it would be time consuming. I would like to group them by topic, so i have first considered LDA for topic classification, then k-means clustering. Is it a good approach to proceed? What are the differences and how I can test the accuracy of the classification?
2
-
1$\begingroup$ I am no expert, in fact I am learning about k-means as we speak. Having said, k-means is unsupervised learning, meaning you have no labels for the data. Therefore it is the right approach for your data. However, this means you cannot measure accuracy of classification, since no labels or involved. There is a score called silhouette which measures the fit of the data to varied number of clusters. $\endgroup$– Borut FlisCommented Jun 22, 2020 at 9:54
-
$\begingroup$ Thank you so much Borut. I am trying to use k-means for grouping tweets and hopefully labelling them. Do you think it could be something doable? $\endgroup$– MathCommented Jun 22, 2020 at 12:50
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
First of all, you use two terms Clustering and Classification interchangably and I would like to draw your attention to this. Your problem is purely Clustering.
Secondly, you asked for testing accuracy. As your problem is pure Clustering, there is no evaluation for that.
The last but not least is the problem of "Short Text Understanding". In short texts, LDA of TF-IDF based approaches (like LSA) do not work well as they rely on co-occurrences of words in texts.
Considering these two facts let's discuss the solution. I would recommend that you use a pre-trained model (I recommend S-BERT which is implemented in Sentence-Transformer Python package). Simply follow this semantic similarity search piece of code and you can implement it easily in a few lines.
Probably fine-tuning the model will be tricky according to the nature of your task, so just use a pre-trained model and see how it works.
For evaluation, I recommend to capture some similar tweets manually (more the better) and check the performance on them.
For Topic Modeling approach you can use the pytho implementation of the paper I mentioned above here.
Two above mentioned can also be combined creatively (search similarity with S-BERT and compare to bi-term topic model for example)
Hope it helped.
Good Luck!
answered Jun 23, 2020 at 9:30
$\begingroup$
$\endgroup$
You can use k-means. It belongs do the domain of unsupervised learning, which means you do not have labels to test classification of examples. You can evaluate what the clusters, defined by k-means have in common and label them accordingly. In the k-means clustering you can assign the number of clusters(k) arbitrarily or evaluate the different number of cluster through silhouette score. A good way to evaluate the clusters the algorithm defined is through visualization.
The choice of what to do depends on what type of attributes you have. K-means tries to minimize the distances between the clusters, so you need continuous attributes. Furthermore, for the purpose of explanation you are better off with less attributes.
