# What are some best Text Representation techniques in NLP

I've studied about various text representation techniques like : Bag of Words, N-gram data modelling, Tf-idf, word embedding etc.

I would like to know which among all the techniques are most efficient when it comes to data modelling or representation for a supervised text classification across a large number of categories.

I might have around 40 categories and then around a same number of sub-categories upto 4 levels.

There's no simple answer to this question. As far as I know in general the choice depends mostly on the type of classification:

• Bag of Words (usually with tf-idf weights) is a simple but quite efficient representation for classification based on the text topic or similar, assuming the classes are reasonably distinct from each other.
• Word embeddings are a more advanced option for semantic-based classification. They can handle more subtle semantic relations but require being trained on a large training corpus. Using pre-defined embeddings can be a solution but then there's the risk that the original training data isn't perfectly suitable for the dataset.
• N-grams models can be used in many different ways but are often chosen when the classification involves syntax and/or writing style. Note that the higher the value $$n$$, the larger the training corpus needs to be, this can also be taken into account in the choice.

I might have around 40 categories and then around a same number of sub-categories upto 4 levels.

It depends on the data but 40 classes is already a very challenging classification task. For the sake of simplicity let's assume a uniform distribution over classes: a random baseline accuracy would be 1/40 = 2.5%. Of course it depends on the data and a good classifier will do better than that, but don't expect too much...

Now 4 levels of 40 sub-categories means 40^4 = 2.5 millions classes! Even assuming you have enough data (say around 10 instances by class in average, that is 25 millions instances!), it's very unlikely that a classifier will be able to predict anything useful from such a huge amount of classes.

• What if I start with around 130 classes, what approach should I use to move forward? Actually, I'm new to NLP. Although I'm studying theory and well understanding it but facing trouble when it comes to deciding an approach. – mercury-01 Jan 28 at 6:26
• @mercury-01 in my experience it's better to start simple, here I would say with tf-idf vectors and something like decision trees for instance (decision trees have the advantage that you can see how the model makes a decision). Then you can analyze what happens specifically with the data and decide which direction to go next. – Erwan Jan 28 at 12:10
• Thanks @Erwan, I'll start with this algorithm and share the results if I found something benefitial. – mercury-01 Jan 29 at 5:17