7
$\begingroup$

I have a large text dataset clusterized. Each cluster is represented by a centroid of the vectorized texts that belong to it, the number of texts, the created date, and other parameters. I can't plot the clusters in an n-dimensional space. Which options do I have?

$\endgroup$
3
  • $\begingroup$ Search for holoviews $\endgroup$
    – Aditya
    Commented Apr 24, 2018 at 5:26
  • $\begingroup$ that bib is great @Aditya! $\endgroup$ Commented Apr 24, 2018 at 13:53
  • $\begingroup$ A sample plot using datashader from the same.. datascience.stackexchange.com/a/28659/35644 @Federico Caccia $\endgroup$
    – Aditya
    Commented Apr 24, 2018 at 14:29

3 Answers 3

3
$\begingroup$

T-SNE is another dimensionality reduction algorithm not mentioned in the article in the other answer. Used for VERY high dimensional data, if you have trained some embeddings for your dataset. Reference Here . Python standard library here.

cheers

$\endgroup$
0
3
$\begingroup$

Several options:

  • Locally Linear Embedding (LLE): This method construct a set of local geometric patches on each of which a data point is reconstructed through the weighted sum of its K nearest neighbor and maps these patches into a lower dimensional space. Find the code here and I strongly advice to use both LLE and Modified LLE and use the better one (visually).
  • t-SNE: Maps the similarity of points in high-dimension, into a low dimensional manifold extracted by their distance in t-distribution. Be careful to tune parameters properly.
  • Spectral Embedding: Spectral Clustering in fact (or to be precise, Spectral Clustering is indeed a simple clustering on Spectrally Embedded version of data). It projects the data on eigenvectors of its Laplacian according to the magnitude of corresponding eigenvalues.

an many more ...

$\endgroup$
1
$\begingroup$

You can use a dimensionality reduction algorithm (like principal component analysis) to reduce the number of dimensions of the data to 2 or 3, and then perform scatter plots using the reduced variables, coloring them according to the cluster they belong to. In this blogpost a similar thing is done.

$\endgroup$
1
  • $\begingroup$ I have thousands of dimensions, so PCA could become in too much loss of information. And I just want to plot the centroids of each cluster. I think the best way is to plot creation time vs another feature, and give to the point(bubble) a radius proportional to the amount of elements in this cluster. $\endgroup$ Commented Apr 23, 2018 at 21:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.