How synthetic sampling of data helps classifier to predict better result?

Question

I am currently working on data imbalance using SMOTE for binary and other algorithms for the multi-class problem.

I have the idea how to create the synthetic example to bring noticeable accuracy on a given dataset.

I want to go into deep and understand how a classifier, especially SVM handle the data with the synthetic example to classify more accurately. It would also helpful to know for other techniques like boosting algorithms, random forest etc.

Any kind of guidance on above question will be very much helpful.

I have already asked this question on StackExchange here but didn't get any answer.

Answer 1

I think that in order to understand how the SVM handles the new synthetic data, you should look at the loss function SVM uses, i.e. hinge loss and the behavior on an imbalanced dataset. Intuitively this function will try to fit the hyperplane that best separates the data. For example imagine you have a dataset that is not linearly separable and you have to classes A (with a 1000 samples) and B (with 50 samples). So the SVM will "move" the hyperplane to classify right all samples of A even if this means failing to classify correctly some samples of class B.Then when you create new samples of the classes B the classifier will try to find a balance between the error of the both classes. A more detailed explanation can be found in the 6.3 section fo this article

In general all classifiers tend to classify correctly the most dominating class because of the loss function they use. One way of dealing with it is generating synthetic data, other way is using a cost sensitive classifier, this is the cost of misclassification of the minority class is higher than the majority class. More details can be found in the following links:

https://svds.com/learning-imbalanced-classes/

http://machinelearningmastery.com/tactics-to-combat-imbalanced-classes-in-your-machine-learning-dataset/

Answer 2

How about using a GAN ( Generative Adversarial Network) to generate undisdinguishable data for your imbalanced dataset. An example for this can be seen here: (https://github.com/osh/KerasGAN).