I have a bag-of-words consisting of about 60000 features. Each feature represents a diminsion. I want to represent this bag-of-words in a reduced 2D space. How do I do it?

I have see an example here, which looks more like what I want but not really the same. In the example they have 2 transform and I only have one. Therefore as suggested I do not want to use pipeline. Below is my code which taking for ever and does not show any error message:

#myList contents about 800000 words
bag_of_words = vec.fit_transform(myList)
X = bag_of_words.todense()
pca = PCA(n_components=2).fit(X)
data2D = pca.transform(X)
plt.scatter(data2D[:,0], data2D[:,1])

I have not found any better option and right now it looks like I am doing something wrong.

What is the best way to visualize a bag-of-words in a scatterplot?

The bag_of_words looks like this:

(0, 548)    3
(0, 4000)   6
(0, 15346)  1
(0, 23299)  1
(0, 22931)  2
(0, 32817)  1
(0, 51733)  1
(0, 38308)  6
(0, 14784)  1
(0, 146873) 1
  • $\begingroup$ Is the PCA visualization not good enough for you? $\endgroup$ – Armen Aghajanyan Dec 8 '16 at 19:11

I think you should have a look at t-SNE which is a visualization technique based on dimensionality reduction. Wikipedia excerpt:

t-distributed stochastic neighbor embedding (t-SNE) is a machine learning algorithm for dimensionality reduction developed by Geoffrey Hinton and Laurens van der Maaten. It is a nonlinear dimensionality reduction technique that is particularly well-suited for embedding high-dimensional data into a space of two or three dimensions, which can then be visualized in a scatter plot. Specifically, it models each high-dimensional object by a two- or three-dimensional point in such a way that similar objects are modeled by nearby points and dissimilar objects are modeled by distant points.

enter image description here

This paper describes a python implementation of the technique on various word2vec datasets.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.