Can we combine multiple K-Means Models as a single model?

Question

I have a NLP problem statement where I use a Word2Vec embedding pre-trained model to convert key text to vectors and then on a set of terms run k-means clustering to get a final model for certain k

For various sets of terms, I would develop a different model, which I would store to disk.

My question is, in case there is a new term, which I wish to classify as to which cluster should it point to from all the models can I follow the following approach?

1. Load all models to memory and get their cluster centers.
2. get the vector of the new term based on the same pre-trained model as before.
3. get distance from each cluster center to the new vector and whichever is nearest can be considered as the winning cluster

I would like to know what could be the possible drawbacks of such an approach.

My assumption is that since the vector space is same as defined by the pre-trained model, therefore the cluster centers would be in the same space.

Answer 1

I agree with your assumption, the vector space is the same so I don't see any major problem with this approach. Still this approach might cause some more subtle bias, depending on the differences between the models (sets of terms, number of clusters). I could imagine the following problems happening:

• if there is a big difference in number of clusters between models, a model which has more clusters is more likely to contain the closest match, simply because it has more centroids. This might favour the most precise clusters (this might actually be a good thing, depends).
• if there are many models sometimes there might be many close centroids across the models, and this would probably make the selection of the closest among them almost random: the exact position of a centroid is significant with respect to other centroids within the same model, not necessarily with respect to other centroids outside the model.