2
$\begingroup$

I have a NLP problem statement where I use a Word2Vec embedding pre-trained model to convert key text to vectors and then on a set of terms run k-means clustering to get a final model for certain k

For various sets of terms, I would develop a different model, which I would store to disk.

My question is, in case there is a new term, which I wish to classify as to which cluster should it point to from all the models can I follow the following approach?

  1. Load all models to memory and get their cluster centers.
  2. get the vector of the new term based on the same pre-trained model as before.
  3. get distance from each cluster center to the new vector and whichever is nearest can be considered as the winning cluster

I would like to know what could be the possible drawbacks of such an approach.

My assumption is that since the vector space is same as defined by the pre-trained model, therefore the cluster centers would be in the same space.

$\endgroup$
1
$\begingroup$

I agree with your assumption, the vector space is the same so I don't see any major problem with this approach. Still this approach might cause some more subtle bias, depending on the differences between the models (sets of terms, number of clusters). I could imagine the following problems happening:

  • if there is a big difference in number of clusters between models, a model which has more clusters is more likely to contain the closest match, simply because it has more centroids. This might favour the most precise clusters (this might actually be a good thing, depends).
  • if there are many models sometimes there might be many close centroids across the models, and this would probably make the selection of the closest among them almost random: the exact position of a centroid is significant with respect to other centroids within the same model, not necessarily with respect to other centroids outside the model.
| improve this answer | |
$\endgroup$
  • $\begingroup$ I feel that the only problem would be if 1 model shares an overlapping cluster with another model...As in 2 doc sets sharing the same vector space, or such a large vector space, that the clusters of 1 model may be engulfed by 1 cluster of another model...and that may lead to the larger cluster never being selected... would that be a problem..?? $\endgroup$ – Fr_nkenstien Oct 11 at 6:49
  • $\begingroup$ Well yes, my answer is essentially about this problem (2nd point): the clusters are comparable because they are in the same vector space, but that means that clusters overlap each other across models. Keep in mind that in K-means clusters are only defined by their closest centroid, there are no bounds to decide what is inside or outside a cluster. It might be ok if the sets of terms used to generate the clusters are very distinct from each other, it depends on the data. $\endgroup$ – Erwan Oct 11 at 9:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.