3
$\begingroup$

I am working on my first ever ds problem on classification. I have got end to end process but not much visualization except bar charts and pie charts.

I want to see how well classes are distinguished in sample dataset itself to find anomaly in sample data or to see how many outliers sample have.

I see many nice scatterplot clustering done in many articles, pappers and books which are mostly based on numerical features. Something like following taken from scikit learn : http://scikit-learn.org/stable/auto_examples/datasets/plot_random_dataset.html

enter image description here

However, most of those are for numeric features. How can I draw similar plot for textual features?

One way I can do it is converting all features into vectors; plot all the indexes(token indexes) on x-axis and their respective metric (TF or TF-IDF) on y-axis? But then there are so many features!?

Also, how can I analyze raw sample data in such clustering format without transforming them into vectors. I can do tokenization and some basic normalization.

For example my sample data is text, category

"chiense restarant near me", chinesefoodlover
"japanese food near me", japanesefoodlover

I can break it down to

[chiense, restarant, near, me], chinesefoodlover
[japanese, food, near, me], japanesefoodlover

But then how to draw clusters of classes from there.

$\endgroup$
  • $\begingroup$ You can also visualize the confusion matrix. You are looking for a diagonal matrix, ideally. $\endgroup$ – Emre Apr 3 '17 at 17:44
2
$\begingroup$

You should look at t-SNE for visualising high dimensional datasets such as words : http://lvdmaaten.github.io/tsne/

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.