What is the correct way to call Keras flow_from_directory() method?

Question

In the following article there is an instruction that dataset needs to be divided into train, validation and test folders where the test folder should not contain the labeled subfolders. Instead it should only contain a single folder (i.e. Test_folder).

When I use the following code, I get the output message refering that no image were found.

Ver.1:
test_generator = test_datagen.flow_from_directory(
    "dataset\\test\\test_folder\\",
    target_size=(IMG_WIDTH, IMG_HEIGHT),
    batch_size=1,
    class_mode=None,
    shuffle=False,
    seed=10)

Output message: "Found 0 images belonging to 0 classes.".

Instead, if I use the same folder structure (dataset\test\class_a\test_1.jpg etc) as in the train and validation folders, everything seems to be OK and I manage to evaluate my model.

Ver.2:
test_generator = test_datagen.flow_from_directory(
    "dataset\\test\\",
    target_size=(IMG_WIDTH, IMG_HEIGHT),
    batch_size=32,
    class_mode='categorical',
    shuffle=False,
    seed=10)

Output message: "Found 1500 images belonging to 3 classes.".

I also tried the recommendation where 'classes' attribute is specified but still 0 images were found.

Ver.3:
test_generator = test_datagen.flow_from_directory(
    "dataset2\\test\\test_folder\\",
    target_size=(IMG_WIDTH, IMG_HEIGHT),
    batch_size=1,
    classes=['test'],
    class_mode=None,
    shuffle=False,
    seed=10)

Output message: Found 0 images belonging to 1 classes.

Thus, what is the correct way to call flow_from_directory() method and why am I getting the message that no files were found? Is my model not correctly evaluated when I use the Ver.2 solution?

Answer 1

Please find a working solution here.

The generators look like:

# Data generators
train_datagen = ImageDataGenerator(
      rescale=1./255,
      rotation_range=40,
      width_shift_range=0.2,
      height_shift_range=0.2,
      shear_range=0.2,
      zoom_range=0.2,
      horizontal_flip=True,
      fill_mode='nearest')

# Note that the validation data should not be augmented!
test_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
        # This is the target directory
        train_dir,
        # All images will be resized to 150x150
        target_size=(150, 150),
        batch_size=batch_size,
        # Since we use categorical_crossentropy loss, we need binary labels
        class_mode='categorical')

validation_generator = test_datagen.flow_from_directory(
        validation_dir,
        target_size=(150, 150),
        batch_size=batch_size,
        class_mode='categorical')

Make sure you have your images stored in the correct way. E.g.

...images/train/class1/
...images/train/class2/
...images/val/class1/
...images/val/class2/

The generator function does need this structure. So make sure there are subfolders for each class in the train/test directories.

You can also use the data generator function for prediction as described in this post.

Answer 2

The above-mentioned scenario(Peter provided) assumes that validation_dir is a parameter of the function of test_datagen.flow_from_directory(). So the logic is that the test_dir just has the one-one folder structure(./test/folder/image) that you provided in the weblink.

test_datagen.flow_from_directory( validation_dir,...) is a method cascading that is syntax which allows multiple methods to be called on the same object. In this way, you can use the function of flow_from_directory().

Answer 3

It can’t find any classes because test has no subdirectories. Without classes it can’t load your images, as you see in the log output above. There is a workaround to this however, as you can specify the parent directory of the test directory and specify that you only want to load the test “class”:

datagen = ImageDataGenerator()
test_data = datagen.flow_from_directory('.', classes=['test'])