Can we combine multiple K-Means Models as a single model?

I have a NLP problem statement where I use a Word2Vec embedding pre-trained model to convert key text to vectors and then on a set of terms run k-means clustering to get a final model for certain k

For various sets of terms, I would develop a different model, which I would store to disk.

My question is, in case there is a new term, which I wish to classify as to which cluster should it point to from all the models can I follow the following approach?

1. Load all models to memory and get their cluster centers.
2. get the vector of the new term based on the same pre-trained model as before.
3. get distance from each cluster center to the new vector and whichever is nearest can be considered as the winning cluster

I would like to know what could be the possible drawbacks of such an approach.

My assumption is that since the vector space is same as defined by the pre-trained model, therefore the cluster centers would be in the same space.