I asked this question on the Statistics SE, but there were no answers, even when a modest bonus was available, so I am asking here to see if any examples can be given.
I have been looking into the imbalanced learning problem, where a classifier is often expected to be unduly biased in favour of the majority class. However, I am having difficulties identifying datasets where class imbalance is genuinely a problem and furthermore, where it is actually a problem, that it can be fixed by re-sampling (e.g. SMOTE) or re-weighting the data.
Can anyone give reproducible examples of real-world (preferably not synthetic) datasets where re-sampling or re-weighting can be used to improve the accuracy (or equivalently misclassification error rate) for some particular classifier system (when applied in accordance with best practice)? This must be an improvement in accuracy on the original data distribution, not the resampled one, as that reflects operational conditions where the classifier will be deployed.
I am only interested in accuracy as the performance measure. There are some tasks where accuracy is the quantity of interest in the application, so I would appreciate it if there were no digressions onto the topic of proper scoring rules, or other performance measures.
It is not an example of the class imbalance problem if the operational class frequencies are different to those in the training set or the misclassification costs are not equal. Cost-sensitive learning is a different issue.
UPDATE: While the answer that received the bounty was not ideal (as it didn't appear to apply the classifier in accordance with best practice), I may well give a new bounty to answers that more fully address the question.