While working with 3 classes,my dataset contains different proportions of all classes. For example,

Class "0"  has  11098 cases.....Normal
class  "1" has  2369 cases....."Abnormal"
class  "2" has  3396 cases....."May be Normal or Abnormal"

I trained the model with similar proportions. I have two questions:

  • Is it necessary to balance all three classes?

  • What is the impact of class imbalance on classification performance?

  • $\begingroup$ Your data is fairly balanced. Don't worry about it. If you are, try adjusting the decision threshold or loss function. I'm more concerned about your representation; is "May be Normal or Abnormal" one of the classes? Are you using some other signal to infer the ground truth; what's going on? $\endgroup$ – Emre May 24 '17 at 6:10
  • $\begingroup$ @Emre...Yes...I extracted my dataset from signals...to see the faulty/No faulty status $\endgroup$ – Case Msee May 24 '17 at 7:09

As mentioned in the comments, your data doesn't have that much of an imbalance. A good way to check how the imbalance is affecting your model, is by calculating precision and recall. These values will be quite low while your accuracy will shoot high because it will predict the majority class for every sample. However the 'maybe normal' case is dangerous. It would be better to set a different threshold , probably a lower one so that even the marginally abnormal cases will be classified as abnormal. A precision recall curve or a ROC curve is a good way to decide on the cutoff.

| improve this answer | |

Do you have data imbalance? The answer is yes.

Is it a problem? It depends.

Your data imbalance ratio is quite high (1:10), is it a problem or not, it depends on three things: - what is the used classifier? - are these separable classes or not ? - are you interested in overall accuracy or average accuracy ?

The impact is to have bias classifier! If you have a classifier that predicts all the samples as class one , you will have overall accuracy of ~ 70%. But it is a biased classifier the results in average accuracy of ~ 33%

| improve this answer | |
  • $\begingroup$ 1:10 is not a high class imbalance. $\endgroup$ – Matthew Drury Jul 26 '17 at 0:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.