I'm trying to play around with unsupervised NLP using Word2Vec. So far, the data i used is very small, but that is because I am just testing to see how Kmeans will work.
The Kmeans was performed first (4 clusters) due to the small number of inputs, and the TSNE was used to visualise to 2D:
model = Word2Vec(sents,
min_count=5,
window=5,
alpha=0.03,
min_alpha=0.0007,
sg=1,
)
model_K = KMeansClusterer(4, distance=euclidean_distance, repeats=50) #cosine distance didnt change as much
assigned_clusters = model_K.cluster(model.wv.vectors, assign_clusters=True)
tsne = TSNE(n_components=2, random_state=0)
vectors = tsne.fit_transform(model.wv.vectors)
As you can see the clustering kind of works, but there are some clusters way off. I'm wondering is that is because I performed the cluster before the reduction in dimensions. But from what I have read it's better to do Kmeans before if you can.
When I try with 6 clusters I get:
Any reasons why the clustering isn't working as expected will be appreciated. Thanks.