# Unsupervised Text Classification with Python: Kmeans

I am working on a project to build a text classifier of questions being asked. There are no labels provided in my data so I have chosen to go with an unsupervised approach. This solution needs to read a new question, and determine which category of classification it falls into

I have been working on a KMeans model, but despite everything I try I just cannot get any decent results. I have even hand selected a data set, cleaned it further by hand to make it as straightforward as possible, and my clusters are still overlapping, and I am getting a low silhouette score of 0.26015371238537227 (this is the highest I've been able to get). My raw data isn't very great to work with, but I still don't understand how it performs so poorly when I hand select/clean the best examples to use as a test. I have included my code for kmeans section below.

Does anyone have any suggestions? I'm a kind of at a loss here. Are there any other good ways to classify unlabeled text? I'm also relatively new to this type of work and this is my first ML based project too so I'm sure there is some knowledge gap to account for as well. I'm about ready to give up on the unsupervised approach and do some manual work to label the data myself and build a supervised model. Figured I'd check here first to see if all hope is lost or not. Any help is greatly appreciated!

    data = pd.DataFrame(text)
data.columns = ['KM']
tfidf = TfidfVectorizer(max_df=0.80, min_df=5, max_features=10000)
text = tfidf.fit_transform(data['KM'].values.astype('U'))

kmeans = KMeans(n_clusters=8, init='k-means++', max_iter=300, n_init=10,
random_state=20)
kmeans.fit(text)

clusters = kmeans.predict(text)

pca = PCA(n_components=2)
two_dim = pca.fit_transform(text.todense())

scatter_x = two_dim[:, 0]
scatter_y = two_dim[:, 1]

plt.style.use('ggplot')

fig, ax = plt.subplots()
fig.set_size_inches(20,10)

cmap = {0: 'green', 1: 'blue', 2: 'red', 3: 'orange', 4: 'black', 5: 'purple', 6:
'yellow', 7: 'pink'}

for group in np.unique(clusters):
ix = np.where(clusters == group)
ax.scatter(scatter_x[ix], scatter_y[ix], c=cmap[group], label=group)

ax.legend()
plt.xlabel('PCA 0')
plt.ylabel('PCA 1')
plt.show()

order_centroids = kmeans.cluster_centers_.argsort()[:, ::-1]

terms = tfidf.get_feature_names()
for i in range(8):
print("Cluster %d:" % i, end='')
for ind in order_centroids[i, :10]:
print(' %s' % terms[ind], end='')
print()

score = silhouette_score(text, labels=kmeans.predict(text))
print(score)

• Welcome to DataScienceSE. Imho you're using the wrong tool: it's not because you don't have labels that you should use unsupervised. Clustering is useful for grouping instances by similarity when one doesn't have any expectations about the resulting groups. If you know the classes that you want to obtain, you need a classification method. It's like saying I don't have a boat so I'm going by car: if the destination is an island it's not gonna work ;) Mar 16, 2021 at 22:40
• Hey! So would you suggest doing some manual effort to classify some training data by hand for use in a different model? Or so you have any suggestions for some other approaches to try? I've been looking at some LDA topic modeling as well but not sure how well-suited that is for providing a classification for each record of text
– Jim
Mar 18, 2021 at 1:39
• Hi Jim, LDA is certainly a better option for clustering text by topic, in general this would work better than k-means and it provides you with a list of top words for each cluster/topic. However it's still unsupervised so it's not sure that the topics will correspond exactly to what you want. Note that you can play with the number of clusters/topics parameter: with a quite high value you might be able to label some of the clusters/topics manually, so essentially label a large group of instances at once. Then you could use this as training set in a supervised classifier to label the ... Mar 18, 2021 at 11:31
• ... rest of the instances, as some of the clusters/topics probably won't correspond to any of your classes. How well this would work depends on your data, but I think it's a reasonable balance between reliability of the predictions and annotation effort. Of course the ideal case for supervised classification is to have a large sample of manually annotated instances, but this is not always practical. Mar 18, 2021 at 11:32