1
$\begingroup$

flow_from_directory in Keras requires images to be in different subdirectories. However, I have the images in a single directory with a csv file specifying the image name and target classes.

How do I use flow from directory directly from csv files named train.csv and test.csv?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Then create diferent directories using Python's OS Module mapped acccording to the CSv's $\endgroup$
    – Aditya
    Mar 2, 2018 at 3:07
  • $\begingroup$ @Aditya would be great if you can give me an example, too blank in that, dunno where to start :( $\endgroup$
    – Srihari
    Mar 4, 2018 at 9:03
  • $\begingroup$ Can you add a screen shot of your CSV file? Your problem is similar to this Competition kaggle.com/c/plant-seedlings-classification/data $\endgroup$
    – Aditya
    Mar 4, 2018 at 9:22
  • $\begingroup$ @Aditya i got two columns in the csv file, "image_index"- contains name of the image eg. "image000001.png" Second column "label"- contains the output class as text $\endgroup$
    – Srihari
    Mar 4, 2018 at 9:25

3 Answers 3

Reset to default
2
$\begingroup$

Something Like This Should Do The Job

When you are doing something new, Mistakes are likely.. Use At Your own Risk Or Try It Out on A Sample And Try it on a Seperate Directory Completely

import pandas as pd
import os
import numpy as np
import shutil

# source is the current directory
# Open dataset file
dataset = pd.read_csv('dataset.csv')
file_names = list(dataset['filenames'].values)
img_labels = list(dataset['labels'].values)

folders_to_be_created = np.unique(list(dataset['labels'])).values

source = os.getcwd()

for new_path in folders_to_be_created:
    if not os.path.exists(".//" + new_path):
        os.makedirs(new_path)

## Be sure that there is nothing else in your directory except the data, csv and the code file, IT's Better to only have your data in that directory and reference the CSV file from a different Directory...

folders = folders_to_be_created.copy()

for f in range(len(file_names)):

  current_img = file_names[f]
  current_label = img_labels[f]

   ## **Check this Line Accordingly** 

  shutil.move("path//to//current//file", "path//to//new//destination//folder//current_label//")
Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you. So the python batch generator or a custom batch generator also does a similar job? I was looking at the util sequence keras.io/utils which generates batches. But still can't figure out how that one works from directories. Your solution is really helpful thoe, thanks :) $\endgroup$
    – Srihari
    Mar 4, 2018 at 10:37
2
$\begingroup$

use flow_from_dataframe

import pandas as pd
data = pd.read_csv("filename.csv") 
df["category"] = df["category"].replace({0: 'cat', 1: 'dog'}) 
train_df, validate_df = train_test_split(df, test_size=0.20, random_state=42)
train_df = train_df.reset_index(drop=True)
validate_df = validate_df.reset_index(drop=True)

train_datagen = ImageDataGenerator(
    rotation_range=15,
    rescale=1./255,
    shear_range=0.1,
    zoom_range=0.2,
    horizontal_flip=True,
    width_shift_range=0.1,
    height_shift_range=0.1
)

train_generator = train_datagen.flow_from_dataframe(
    train_df, 
    "../input/train/train/", 
    x_col='filename',
    y_col='category',
    target_size=IMAGE_SIZE,
    class_mode='categorical',
    batch_size=batch_size
)
Improve this answer
$\endgroup$
1
$\begingroup$

Try this:

training_set = train_datagen.flow_from_directory(training_path,
                                             target_size = (64, 64),
                                             batch_size = 32,
                                             class_mode = 'binary')


imgs, labels = next(training_set)
Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ This won't work as the images are not in different directories but in a single directory mapped by a CSV file, so we need to first seperate them and then this will probably work... $\endgroup$
    – Aditya
    Apr 8, 2018 at 4:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.