Uneven dataset, how I should train the model if I can't load all pictures at once

I have a dataset of multiple classes (About 50). The dataset doesn't have the same number of pictures per class, some have 300, some have 1000, and some more, and I've seen that this ruined my accuracy on my model

First of all, the data is too big for me to store in the RAM, so I have to split it into parts (I take 300 pictures of each class at once), train the network on that data then repeat the process until I have no more photos left.

The question I have is, is it a problem if I do this (I guess it is from my results)? I have for example from 100 classes of pictures, only 10 left and I train the model only on those 10,then on only 5,then on 3,etc.? Because I did that and after I went over the smallest classes, the overall accuracy went up, but those smaller classes accuracy went down, and in the end from 80% in the beginning for each class I got 0-2% on 48 classes, and 99% on 2 classes.

How can I solve this 'unevenness' (I don't know the word) then so I won't have this problem anymore?

From your tags, I see that you use keras. Keras offers you the class ImageDataGenerator which has the method flow_from_directory() (see here). This method loads the images in your training directory batch by batch from the hard drive and stores only the current batch in the RAM. This eliminates the bottleneck you currently face when loading the images.

To tackle the class imbalance, the recommended method is to use the class_weight argument of the keras classifiers. This argument assigns a weight to every class in your data, allowing you to give higher importance to images from the minority classes. This answer shows how you can calculate the class weights.

In the code below, I put that all together:

# Define constants - change them according to your requirements
BATCH_SIZE = 128
EPOCHS = 50
IMAGE_SIZE = 224

# Set up Image Data Generator
train_datagen = ImageDataGenerator(dtype=np.float16) # here you can also do some data augmentation

# Set up flow from directory
train_generator = train_datagen.flow_from_directory(directory="path/to/your/directory",
class_mode="categorical",
target_size=(IMAGE_SIZE, IMAGE_SIZE),  # resize the images if required
batch_size=BATCH_SIZE)

# Calculate class weights
counter = Counter(train_generator.classes)
max_val = float(max(counter.values()))
class_weights = {class_id: max_val/num_images for class_id, num_images in counter.items()}

#
# Here you set up your model ...
#

# After compiling the model, you fit it to your data using fit_generator
model.fit_generator(train_generator,
steps_per_epoch=train_generator.n // BATCH_SIZE,
epochs=EPOCHS,
class_weight=class_weights,  # use the class_weights as method parameter
verbose=1)


The problem with your approach is that at some point, your network never gets examples with the rare classes. As a result, it does not get penalized if he makes an update that reduces the accuracy on the small size classes (since there are no examples left).

You can think or several ways to circumvent this issue:

1. You can resample (with replacement) your smaller classes so that you have the same number of example in each class. You can then shuffle your resampled dataset and use that as training.

2. You can have a weighted loss with a higher weight on rare classes (the network is more penalized when it makes an error on rare classes). However this might be harder to train since it would produce bigger gradients.

But in general, I think you should just shuffle your whole dataset, not (if I understood correctly) take the same number of each and go over the remaining classes in the end. It definitely would add a bias towards these last classes.