2
$\begingroup$

1 Is there a way to handle data imbalance? ie if data in each class for training is not balanced, say some classes have 50 documents some other have 200 documents. How to handle this?

2 How to handle the classification problem with a large number of classes? I have around 50+ classes (may increase once more data is available) to learn. I am trying out different algorithms and features. Is there any way to handle classification problem with a large number of classes?

$\endgroup$
  • $\begingroup$ I have a question, post completion of your transformation of documents everything will be the format of rows right? $\endgroup$ – Toros91 Nov 15 '17 at 9:12
  • $\begingroup$ post completion of your transformation - you mean vectorization right? $\endgroup$ – Sreejithc321 Nov 15 '17 at 14:26
  • $\begingroup$ yes, after that everything will be the form of records ? $\endgroup$ – Toros91 Nov 16 '17 at 4:05
6
$\begingroup$

People talk a lot about data imbalance, but in general I think you don't need to worry about it unless your data is really imbalanced (like <1% of one label). 50/200 is fine. If you build a logistic regression model on that dataset, the model will be biased towards the majority class - but if you gave me no information about an input to classify, the prior probability is that the new input is a member of the majority class anyway.

The question you want to be able to answer is whether you are differentiating classes fine - so if you do have a minority class, do NOT use 'accuracy' as a metric. Use something like area under the ROC curve (commonly called AUC) instead.

If your data is really super imbalanced, you can either over-sample the minority class or use something called 'SMOTE', for "Synthetic Minority Over-Sampling Technique", which is a more advanced version of the same thing. Some algorithms also let you set higher weights on minority classes, which essentially incentivizes the model to pay attention to the minority class by making minority-class errors cost more.

To learn to differentiate between lots of classes, I think (a) you will need to have a ton of examples to learn from and (b) a model that's expressive enough to capture class differences (like deep neural network, or boosted decision tree), and (c) use softmax output. If those still don't work, you might try a 'model-free' approach like K-nearest-neighbors, which matches each input to the most similar labeled data. For kNN to work however, you need to have a very reasonable distance metric.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.