1
$\begingroup$

Provided a set ($m$ no. of) of n-dimensional vectors what would be the correct unsupervised approach to cluster them? The vectors essentially represent patterns.

For example: Set of vector is represented as $V$. Let a vector $v1$ represents a pattern similar to $y = sin(x)$ curve. The $y$ values are stored in $v1$ and the $x$ intervals are same for all the vectors. Similarly there is a vector $v2$ representing pattern similar to $y=log(x)$.

The problem: Does a group of vectors exist, exhibiting similar (not exactly same) pattern as $v1$, similarly for $v2$ and so on?

Therefore these patterns are required to be clustered appropriately. There are methods such as Vector Quantization, but I am not sure if those methods are appropriate in this case.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Since Neural Networks can learn any non-linear function, I would go for an Autoencoder for dimensionality reduction first, and then run a k-Means clustering on the encoded features. If a regular (non-linear) pattern in the data exist, an Autoencoder that is "deep enough" should be able to account for that.

For the same reasons (i.e. non-linear patterns) I would avoid linear techniques such as PCA.

If this suits you, you can check my Notebooks. I implemented Autoencoders in TensorFlow 1.x here, and in TensorFlow 2.0 here.

Improve this answer
$\endgroup$
4
  • $\begingroup$ t-SNE is also a non-linear technique for dimensionality reduction. It's already available in sklearn. If you have enough data, nothing matches the power of Neural Network, but its implementation is certainly quicker. $\endgroup$
    – Leevo
    Jun 18, 2019 at 8:06
  • $\begingroup$ So, the purpose of the autoencoder is to reduce the dimension (in this two dimension to a single dimension). Then the encoded data can be clustered with suitable clustering method. Did I understand the approach correctly? $\endgroup$
    – tachyon
    Jun 18, 2019 at 11:42
  • $\begingroup$ Yes, that would be my approach. $\endgroup$
    – Leevo
    Jun 18, 2019 at 12:22
  • $\begingroup$ @Leevo The first comment is "wrong" and better to be fixed/removed. t-SNE is a visualization algorithm and "must not" be used for dimensionality reduction purpose. It is non-parametric and stochastic thus can not be reproduced in same way. UMAP is the right equivalent/alternative for dimensionality reduction. $\endgroup$
    – Kasra Manshaei
    Jul 17 at 8:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.