9
$\begingroup$

I am trying to classify images to more then a 100 classes, of different sizes ranged from 300 to 4000 (mean size 1500 with std 600). I am using a pretty standard CNN where the last layer outputs a vector of length number of classes, and using pytorch's loss function CrossEntropyLoss.

I tried to use $weights = \frac{max(sizes)}{sizes}$ for the cross entropy loss which improved the unweighted version, not by much.

I also thought about duplicating images such that all classes ends up to be of the same size as the larges one.

Is there any standard way of handling this sort of imbalance?

Improve this question
$\endgroup$
1
13
$\begingroup$

If you are looking for just an alternative loss function:

Focal Loss has been shown on imagenet to help with this problem indeed.

Focal loss adds a modulating factor to cross entropy loss ensuring that the negative/majority class/easy decisions not over whelm the loss due to the minority/hard classes.

I would look into using that is it seems to be really promising.

Focal Loss Formula

Link To Focal Loss Paper: https://arxiv.org/pdf/1708.02002.pdf

Improve this answer
$\endgroup$
3
$\begingroup$

To handle class imbalance, do nothing -- use the ordinary cross-entropy loss, which handles class imbalance about as well as can be done. Make sure you have enough instances of each class in the training set, otherwise the neural network might not be able to learn: neural networks often need a lot of data. Assuming you care about global accuracy (rather than the average of the accuracy on each individual class, say), I wouldn't bother with a weighted cross-entropy loss or duplicating images.

Your training sounds rather small. To deal with that, you might try starting from an existing pre-trained model and fine-tune the last few layers. Also use image augmentation.

Improve this answer
$\endgroup$
8
  • $\begingroup$ When writing size I meant the class group size, not the images, the images are obviously all the same size.Maybe I forgot to mention it but I cannot assume the test data is distributed like the training data. The test data is most likely balanced! so If I care about the global accuracy shouldn't the network train on balanced data as well? $\endgroup$
    – Itaysason
    May 16 '18 at 7:02
  • 1
    $\begingroup$ @Itaysason, well, that's a very different question (about test data coming from a different distribution than the training data), and should be asked separately. Generally speaking supervised learning assumes they come from the same distribution, so any difference in distribution requires extra work. Yes, you probably want to augment the training data set so its distribution matches that of the test data set $\endgroup$
    – D.W.
    May 16 '18 at 15:08
  • 2
    $\begingroup$ Ordinary cross entropy loss does not handle class imbalance. Where does this advice come from? Resampling your dataset and class weights are common ways of dealing with imbalanced datasets. $\endgroup$
    – MattSt
    Sep 3 '19 at 14:31
  • $\begingroup$ @MattSt, I'd say that depends what you mean by "handle". If the training and test set come from the same distribution, my impression is that using cross-entropy is often reasonable, with no extra resampling or class weights. (If the training and test sets have the same class, oversampling the minority class may be beneficial in some cases, but it also causes a bias that can make global accuracy worse in other cases. Ideally, one would try both to see which is better.) If the training and test set come from different distributions, see my comment above. $\endgroup$
    – D.W.
    Sep 3 '19 at 17:00
  • $\begingroup$ @D.W. Please explain how does CE handle class imbalance in the training dataset. Seems to me that the -log(y_hat) doesn't take the class distribution into consideration. $\endgroup$
    – Ondrej Sotolar
    Oct 6 at 10:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.