Sometimes we come across datasets where classes are imbalanced. For example, class A may have 2000 instances, but class B has only 200. How can we train a classifier for such datasets?
Well, you just train it on unbalanced dataset, it is not a problem. I don't think you need to apply some special techniques.
The only case where you may want to do something special is if your classes are skewed by their nature (if the skewness is the property of class itself, not of just your specific dataset). For example, if you build a classifier which will tell if someone has a malignant tumor in X-ray image or not. By nature of the task absolute majority of patients will not have a malignant tumor. In such case you may want to tune your classifier algorithm a bit, for example introduce a weighting into your SVM or smth like it.
A special case is if you have EXTREMELY unbalanced classes. For example 100000 of positive examples and 20 negatives. In such case you will want to go away from classification task to the approach called 'anomaly detection'.
There are a few of other approaches you can take to try to balance your class distribution.
Subsample Majority Class
You can balance the class distributions by subsampling the majority class.
Oversample Minority Class
Sampling with replacement can be used to increase your minority class proportion.
A more sophisticated scheme is to add Gaussian, or other suitable, noise to the existing instances of the minority class in an effort to create a greater number of representative, but diverse, instances.
A popular method to synthesise minority class instances with greater complexity than pure noise addition is SMOTE ( Synthetic Minority Oversampling TEchnique ). This uses a K-member neighbourhood in feature space to impute new instances.
WEKA has a filter for this.
Though there is some evidence that this technique is not overly beneficial with high dimensional data here.