I have list of log comments in CSV file. I want to cluster those log comments using K-Means and after that I want convert each cluster comments into general form. for eg. I have bunch of comments in one cluster which starts from "Reservation number failed......" and I want to convert those comments into particular comment like "Reservation failure".

I can achieve this by giving specific name to each cluster after seeing each cluster. But I don't want like this. I want to create intelligent model which automatically create generalized comments for me.

I would not like to assign name to each cluster. Basically I am done with clustering part. that is, I have lets say 3 clusters as below

  • cluster 0 : list of comments like "Reservation number failed......", total comments: 15
  • cluster 1 : list of comments like "Request timeout failed due to ......", total comments:9
  • cluster 2 : list of comments like "Dinning reservation successfully completed...", total comments: 5

I want to build model that intelligently assign name to each cluster by its contents. for eg .

  • cluster 0 will get name as "Reservation failure"
  • cluster 1 will get name as "Request timeout failure"
  • cluster 2 will get name as "Dinning reservation successful"

if after training more data with some different comments. it should create another cluster and assign the name as per content.


Basically, we want to cluster similar comments and assign a name/entity to it. I suggest you to use Doc2Vec to convert the comments to fixed-sized vectors. Each of your comments will then be a n-dimensional vector. Comments with similar words/phrases will lie in the close proximity of one another.

Now, using K-Means Clustering, we can form clusters of vectors that represent comments having a similar meaning. Once the clusters are formed, assign a name to each of them.

For a given sample ( comment ), the model will first transform the given comment into a vector. The model will then check for a cluster that is nearest to the given sample's vector. The output will be the name of the nearest cluster.

  • $\begingroup$ can you please help me with some examples!! $\endgroup$ Mar 6 '20 at 10:59
  • $\begingroup$ This blog might help. $\endgroup$ Jun 28 '21 at 1:13

Not the answer you're looking for? Browse other questions tagged or ask your own question.