Clustering or classifing n-gram-based text categories

Question

I have large set of data records looking like this:

"text", "category"

I extract n-grams from text (2-, 3- and 4-grams) and store count of each n-gram per category, like so:

"ngram1", "category1", 1000
"ngram1", "category2", 20
"ngram1", "category3", 15
"ngram2", "category1", 25
"ngram2", "category2", 550
"ngram2", "category3", 600

Is there a clustering or classification algorithm that could help with finding groups within categories, based on counts of same n-grams within categories?

In the above example that algorithm should be able to show that "category2" and "category3" are part of the same group.

Answer 1

If your ultimate goal is to cluster similar categories and assuming that you have labels of each text as category1, category2,...,categoryN from 1 to N, a bag of words method would be sufficient in order to create features so that you can run multiple desired clustering algorithms.

K-means can be a good starting point for getting similar groups of text document categories, you can try out different k for more sensible outcomes.

Methodology that I would adapt here, forgive me if understood your problem wrong, would be to concatenate all the textual data for a given category and create 1,2,3,..n gram vectors. The values of these vectors can be TF-IDF of a given n-gram, and so on. Still different feature matrices can be formed to compare category vectors but TF-IDF is a pretty common way to do it in.

After acquiring n-gram feature vectors for each text category, you can apply k-means algorithm with several distance metrics to find similar categories.

Suppose we have 5 categories of text as given:

1. Soccer Games
2. Basketball Games
3. Politics
4. Movies

The proposed method can capture the similarity between soccer and basketball games as they are both sports.

Hope this can help

Answer 2

Looking at the problem statement, it seems all you want is to apply Hierarchical clustering algorithm to map sub categories with their category(child-parent relationship).

Clustering algorithm provides you the distance of each cluster with near by cluster and intra-distance within the cluster. This could not probably solve your problem. Go for Hierarchical clustering to map sub-categories with categories.