I'm using the data analysis software Orange to analyze rows of data with labels $\{H, T,L\}$. $T$ is the neutral state of the system I'm trying to model so data is almost always labeled with $T$. This leads to an extremely high $97$% classification accuracy, that is untrue. Almost no datapoint with an $H, L$ actually gets labeled with an $H, L$ respectively by the neural net. The neural net is doing what's logical: flatten everything to $T$ since that occurs most of the time. How can I repair this situation?

  • $\begingroup$ There are downsampling and oversampling approaches to mitigate the imbalance. There are also cost-sensitive approaches, but I'm not sure if they can be applied with neural networks. Also consider computing recall metrics, they should be very low. $\endgroup$ – Gabriel Romon Aug 27 '18 at 8:07

There are a few ways:

  1. Oversample the under-represented class. issue: Leads to very constrained boundaries (not smooth) around the under-represented class

  2. Weigh the loss coming from under-represented class more than over-represented class. If you think about it, this is mathematically doing the same thing as 1

  3. Use SMOTE (here) (i recommend this). Basically samples additional points randomly from lines joining nearest under-represented classes and increases the dataset size accordingly. scikit contrib implementation of smote

There are other ways but they offer very marginal improvement over SMOTE


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.