Kmeans with Word2Vec model unexpected results

Question

I'm trying to play around with unsupervised NLP using Word2Vec. So far, the data i used is very small, but that is because I am just testing to see how Kmeans will work.

The Kmeans was performed first (4 clusters) due to the small number of inputs, and the TSNE was used to visualise to 2D:

model = Word2Vec(sents,
                 min_count=5,
                 window=5,
                 alpha=0.03,
                 min_alpha=0.0007,
                 sg=1,
                 )

model_K = KMeansClusterer(4, distance=euclidean_distance, repeats=50) #cosine distance didnt change as much
assigned_clusters = model_K.cluster(model.wv.vectors, assign_clusters=True)

tsne = TSNE(n_components=2, random_state=0)
vectors = tsne.fit_transform(model.wv.vectors)

As you can see the clustering kind of works, but there are some clusters way off. I'm wondering is that is because I performed the cluster before the reduction in dimensions. But from what I have read it's better to do Kmeans before if you can.

When I try with 6 clusters I get:

Any reasons why the clustering isn't working as expected will be appreciated. Thanks.

Answer 1

The question is what is expected from clustering?

Kmeans works well on gaussian distributed data. In this case both clsuetrings are "right". Some of overlaps comes to the fact that you applied kmeans before visualization and TSNE has to compress some information which is present in high-dimensional data. It means some of overlap points are not overlapping in higher dimensions but you see them overlapping here.

Last but not least is the fact that when clusters are not well-separated, kmaens produces overlapping anyways.