I have a situation where I have to cluster word2vec vectors (200 length dimension vectors on a very large corpus). I decided to use Density based clustering (DBSCAN, HDBSCAN) because my dataset is very high in noise, and I do not want it to be part of my clusters. My knowledge on Cosine distance is limited but I find that Density based clustering algorithms do not have a direct implementation using cosine distance (
pairwise_distance calculation is too memory intensive).
My question here is can I normalize the Word2vec vectors using L2 normalization using:
norm_data = normalize(vector_array, norm='l2') from the sklearn library to normalize these vectors, and then use euclidean distance over the normalized vectors?
Can someone suggest any other better technique to cluster word vectors when there is noise in the dataset?