CNN - imbalanced classes, class weights vs data augmentation

Question

I have a dataset with a few strongly imbalanced classes, eg. the smallest class is about 54 times smaller than the largest. Therefore, data augmentation in order to equalize the size of classes seems like a bad idea to me (in the example above each image would have to be augmented 54 times on average). So I thought that I could do less augmentation of minority classes and then use class weights in the loss function. Is this approach better than the mere augmentation or just the use of class weights?

Answer 1

Is this approach better than the mere augmentation or just the use of class weights ?

Note that data augmentation is the process of changing the training samples (e.g. for images, flipping them, changing their luminosity, adding noise, etc.) and adding them back into the set. It is used for enriching the diversity of training samples, thus, in this aspect it cannot be replaced with class weighting. However, it is related to over-sampling since we can both enrich and increase the size of a class via data-augmentation. For now we are only focusing on the size increasing part, i.e. data over-sampling.

Data-oversampling and class weighting are equivalent. Copying the samples of a class 3X is equivalent to assigning a 3X weight to the class. However, the weighting is better from storage and computational point of view since it avoids working with a larger data-set.

Note that based on this equivalency we can mix and match. Increasing the size of a class 6X $\equiv$ increasing the weight 6X $\equiv$ increasing the size 3X and increasing the weight 2X.

Class weights vs over-sampling in more detail

I copied my answer from here, since the solutions are almost the same but questions are the reverse of each other.

This challenge can be tackled in two places:

1. Data: as you mentioned, this is done by artificially increasing the number of samples for under-represented class $uc$. This produces the same effect as data-sets that are naturally balanced,

2. Model: this is generally done by over-penalizing the miss-classification of $uc$ compared to other classes. One place for this modification is the loss function. A frequently used loss function in classification is cross entropy. It can be modified for this purpose as follows. Let $y_{ik}$ be $1$ if $k$ is the true class of data point $i$, and be $0$ otherwise, and $y'_{ik} \in (0, 1]$ be the corresponding model estimation. The original cross-entropy can be written as: $$H_y(y')=-\sum_{i}\sum_{k=1}^{K}y_{ik}log(y'_{ik})$$
   which can be weighted as $$H_y(y')=-\sum_{i}\sum_{k=1}^{K}\color{blue}{w_{k}}y_{ik}log(y'_{ik})$$ For example, by setting $w_{uc} = 10$ and $w_{k \neq uc}=1$, we are essentially telling the model that miss-classifying $1$ member from $uc$ is as punishable as miss-classifying $10$ members from other classes. This is roughly equivalent to increasing the ratio of class $uc$ $10$ times in the training set using method (1).
   As a concrete example, suppose ratio of three classes are $c_1=10\%$, $c_2=60\%$, and $c_3=30\%$, thus, we can set $w_{c_1}=6$, $w_{c_2}=1$, and $w_{c_3}=2$, which implies $w_{c_i} \times P(c_i)=w_{c_j}\times P(c_j)$ for all class pairs $c_i$ and $c_j$.