# Clustering with arrays / vectors as features?

I am trying to cluster a large set of documents of which I have a DOC2VEC representation. But I want to cluster them with more features, thus resulting in having both a vector (numpy.asarray(doc2vec_for_document)) and single values as features.

Is this possible? I would like to try clustering with both K-means and DBSCAN of the SKLearn module in Python.

My dataset would look like this:
| Document | DOC2VEC | extra_ft1 | extra_ft2 | ...

Also, what if I had multiple vectors instead of 1 vector and many values?

• What do you mean by multiple vectors instead of 1 vector and many values? – David Masip May 29 '18 at 15:22

You need to define a distance function that yields the desirable output.

Usually it will be enough if you can construct it such that d(a,b)for your purpose.

Doc2vec vectors are a bit tricky because they have so many dimensions, and a very strange geometry. It doesn't even seem to be clear whether cosine or Euclidean is to be used on these vectors...

Either way, you have to carefully balance the different features. In the other answer, minmax or stddev scaling was proposed. On one hand, that likely ruins the doc2vec properties, on the other hand this will but much more weight on the word vectors than on the other attributes.

For DBSCAN, you can also follow the "Generalized DBSCAN" approach. Here, the idea is just to define different thresholds for different features. Then neighbors must satisfy all thresholds. I.e. have doc2vec cosine less than A, and other features' distance less than B. This is likely easier than balancing them as factors inside a single distance function.

But nothing saves you from weighting different feature (sets} carefully.

Yes, it is possible. First, you should concatenate the outputs of doc2vec with your extra feature vectors to form an augmented feature vector. Then, you must scale the augmented feature vector, using min-max scaling or standardization (zero mean, unit variance). That is because the different features should be in the same range to have a good behaviour in clustering task. You can find these pre-processing modules in scikit-learn: http://scikit-learn.org/stable/modules/preprocessing.html

• Perfect, so if I had multiple vectors I would just need to concatenate the vectors, thank you – Nicolò Gasparini May 30 '18 at 6:51
• I’m glad that my answer helped you. You’re welcome. – pythinker May 30 '18 at 6:54
• It's not that easy. If you scale the data this way, and use a 300 dimensional doc2vec vector, and 3 extra features, the result will be completely dominated by doc2vec, the extra features will have next to no impact. Don't just do minmax scaling etc. - that assumes every feature has the same importance. – Has QUIT--Anony-Mousse Jun 3 '18 at 7:42