I have about 40 clusters of documents defined by a combination of k-means clustering algorithm and hand curation. For example, some of the clusters given by k-means are too noisy so they have been further subdivided.
Now I want to assign new documents to these clusters.
I found that it is possible to extract keywords using tf-idf based methods as mentioned here.
My approach is to extract key terms from each of these clusters using tf-idf based method and I can extract the keywords from the new document using the same method.
My question is, how do I assign the new document to the cluster that has the most similarity?
Edit: I do not have enough reputation to comment on Marks answer: the input to kmeans are document vectors (from doc2vec) of all documents -- and I get the centroids of the initial clusters i.e. centroids = kmeans_model.cluster_centers_
. But I have split many of these clusters manually into sub clusters. For example, original cluster 3 is now two clusters -- 3_1 and 3_2. How do I generate a representative vectors (like a centroid) for the documents in these sub clusters?