flow_from_directory in Keras requires images to be in different subdirectories. However, I have the images in a single directory with a csv file specifying the image name and target classes.

How do I use flow from directory directly from csv files named train.csv and test.csv?

  • 1
    $\begingroup$ Then create diferent directories using Python's OS Module mapped acccording to the CSv's $\endgroup$ – Aditya Mar 2 '18 at 3:07
  • $\begingroup$ @Aditya would be great if you can give me an example, too blank in that, dunno where to start :( $\endgroup$ – Srihari Mar 4 '18 at 9:03
  • $\begingroup$ Can you add a screen shot of your CSV file? Your problem is similar to this Competition kaggle.com/c/plant-seedlings-classification/data $\endgroup$ – Aditya Mar 4 '18 at 9:22
  • $\begingroup$ @Aditya i got two columns in the csv file, "image_index"- contains name of the image eg. "image000001.png" Second column "label"- contains the output class as text $\endgroup$ – Srihari Mar 4 '18 at 9:25

Something Like This Should Do The Job

When you are doing something new, Mistakes are likely.. Use At Your own Risk Or Try It Out on A Sample And Try it on a Seperate Directory Completely

import pandas as pd
import os
import numpy as np
import shutil

# source is the current directory
# Open dataset file
dataset = pd.read_csv('dataset.csv')
file_names = list(dataset['filenames'].values)
img_labels = list(dataset['labels'].values)

folders_to_be_created = np.unique(list(dataset['labels'])).values

source = os.getcwd()

for new_path in folders_to_be_created:
    if not os.path.exists(".//" + new_path):

## Be sure that there is nothing else in your directory except the data, csv and the code file, IT's Better to only have your data in that directory and reference the CSV file from a different Directory...

folders = folders_to_be_created.copy()

for f in range(len(file_names)):

  current_img = file_names[f]
  current_label = img_labels[f]

   ## **Check this Line Accordingly** 

  shutil.move("path//to//current//file", "path//to//new//destination//folder//current_label//")
  • $\begingroup$ Thank you. So the python batch generator or a custom batch generator also does a similar job? I was looking at the util sequence keras.io/utils which generates batches. But still can't figure out how that one works from directories. Your solution is really helpful thoe, thanks :) $\endgroup$ – Srihari Mar 4 '18 at 10:37

use flow_from_dataframe

import pandas as pd
data = pd.read_csv("filename.csv") 
df["category"] = df["category"].replace({0: 'cat', 1: 'dog'}) 
train_df, validate_df = train_test_split(df, test_size=0.20, random_state=42)
train_df = train_df.reset_index(drop=True)
validate_df = validate_df.reset_index(drop=True)

train_datagen = ImageDataGenerator(

train_generator = train_datagen.flow_from_dataframe(


Try this:

training_set = train_datagen.flow_from_directory(training_path,
                                             target_size = (64, 64),
                                             batch_size = 32,
                                             class_mode = 'binary')

imgs, labels = next(training_set)
  • 2
    $\begingroup$ This won't work as the images are not in different directories but in a single directory mapped by a CSV file, so we need to first seperate them and then this will probably work... $\endgroup$ – Aditya Apr 8 '18 at 4:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.