2
$\begingroup$

I am trying to cluster a large set of documents of which I have a DOC2VEC representation. But I want to cluster them with more features, thus resulting in having both a vector (numpy.asarray(doc2vec_for_document)) and single values as features.

Is this possible? I would like to try clustering with both K-means and DBSCAN of the SKLearn module in Python.

My dataset would look like this:
| Document | DOC2VEC | extra_ft1 | extra_ft2 | ...

Also, what if I had multiple vectors instead of 1 vector and many values?

$\endgroup$
1
  • $\begingroup$ What do you mean by multiple vectors instead of 1 vector and many values? $\endgroup$ May 29, 2018 at 15:22

2 Answers 2

1
$\begingroup$

You need to define a distance function that yields the desirable output.

Usually it will be enough if you can construct it such that d(a,b)for your purpose.

Doc2vec vectors are a bit tricky because they have so many dimensions, and a very strange geometry. It doesn't even seem to be clear whether cosine or Euclidean is to be used on these vectors...

Either way, you have to carefully balance the different features. In the other answer, minmax or stddev scaling was proposed. On one hand, that likely ruins the doc2vec properties, on the other hand this will but much more weight on the word vectors than on the other attributes.

For DBSCAN, you can also follow the "Generalized DBSCAN" approach. Here, the idea is just to define different thresholds for different features. Then neighbors must satisfy all thresholds. I.e. have doc2vec cosine less than A, and other features' distance less than B. This is likely easier than balancing them as factors inside a single distance function.

But nothing saves you from weighting different feature (sets} carefully.

$\endgroup$
-1
$\begingroup$

Yes, it is possible. First, you should concatenate the outputs of doc2vec with your extra feature vectors to form an augmented feature vector. Then, you must scale the augmented feature vector, using min-max scaling or standardization (zero mean, unit variance). That is because the different features should be in the same range to have a good behaviour in clustering task. You can find these pre-processing modules in scikit-learn: http://scikit-learn.org/stable/modules/preprocessing.html

$\endgroup$
3
  • $\begingroup$ Perfect, so if I had multiple vectors I would just need to concatenate the vectors, thank you $\endgroup$ May 30, 2018 at 6:51
  • $\begingroup$ I’m glad that my answer helped you. You’re welcome. $\endgroup$
    – pythinker
    May 30, 2018 at 6:54
  • $\begingroup$ It's not that easy. If you scale the data this way, and use a 300 dimensional doc2vec vector, and 3 extra features, the result will be completely dominated by doc2vec, the extra features will have next to no impact. Don't just do minmax scaling etc. - that assumes every feature has the same importance. $\endgroup$ Jun 3, 2018 at 7:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.