I want to apply unsupervised clustering on a set of short texts, which I need to divide into 2 clusters. Also I know that one of my clusters is likely to contain some words (non-exhaustive list) and I would like to use them for driving my machine learning.

I did not really find any simple way to do it (from any algorithm, library, tool...). Do you know one or have any solution to help me doing this?

I was thinking about applying some kind of k-means on my DTM matrix, using a specific metric e.g the euclidian distance between 2 texts which I would "artificially" reduce when the 2 texts words match my primary list of words which are likely to appear.

I did not test it yet and I was wondering if this solution could be relevant or not.


Your direction of using k-means on term-document matrix is correct. To go further, you can

Use Weighted k-means

You can also try artificially boosting the weight for certain columns/variables (corresponding to your pre-specified list of words). This will force the similarity calculation to be dominated by those words.

Use Constrained k-means

You can read up more about constrained k-means algorithm: you can force certain points to be in the same cluster. In other words, if there are two documents containing words from the pre-specified list, they are forced to be in same cluster.

Alternative: Classification

If you know that there are only two clusters to be formed, why stick to only unsupervised learning? Depending on the application and availability of labelled data, you may think of it as a binary classification task using nearest neighbor or similar algorithm.

  • $\begingroup$ Thanks for your suggestions, will try these asap :-) About classification, my data are not labeled and I have a large amount of these. Maybe I could label some of these and apply semi-supervised methods. $\endgroup$ – TheDahaka Jan 5 '18 at 13:25

K-means doesn't work reliable with such distance hacks. It can prevent the algorithm from converging.

The usual approach would be to rather use this information to guide cluster extraction from a hierarchical clustering. I.e., you would merge clusters if they agree on the desired words, and stop merging if they do not agree, rather than cutting at a single height.

  • $\begingroup$ Thank you for your suggestion. My list is not exhaustive and there are probably a lot of words which are not inside, it is just an additional knowledge. Therefore do you think that hierarchical clustering is still a good option ? $\endgroup$ – TheDahaka Jan 5 '18 at 15:04
  • $\begingroup$ Yes, this approach is meant for incomplete side information. If you had complete word lists, you could use classification. $\endgroup$ – Has QUIT--Anony-Mousse Jan 5 '18 at 19:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.