How do I turn images dataset into a numpy array?

Question

I have a directory for a dataset of images, I I want to transorm it to a numpy array in order to be able to fit an image generator to it. What I have tried to do is the following:

trainingset_temp = '/content/drive/My Drive/Colab Notebooks/Train'
testset = '/content/drive/My Drive/Colab Notebooks/Test'

import cv2
import glob

trainingset = []
files = glob.glob ('/content/drive/My Drive/Colab Notebooks/Train/Haze') # your 
image path
for myFile in files:
    image = cv2.imread (myFile)
    trainingset.append (image)

files = glob.glob ('/content/drive/My Drive/Colab Notebooks/Train/Sunny')
for myFile in files:
    image = cv2.imread (myFile)
    trainingset.append (image)

files = glob.glob ('/content/drive/My Drive/Colab Notebooks/Train/Snowy')
for myFile in files:
    image = cv2.imread (myFile)
    trainingset.append (image)

files = glob.glob ('/content/drive/My Drive/Colab Notebooks/Train/Rainy')
for myFile in files:
    image = cv2.imread (myFile)
    trainingset.append (image)

trainingset = np.array(trainingset,dtype='float32')

but it gives me the following error message:

FileNotFoundError                         Traceback (most recent call last)
<ipython-input-44-2a679dbff06e> in <module>()
     54     batch_size=batch_size,
     55     class_mode="categorical",
---> 56     shuffle=True
     57 )
     58 

1 frames
/usr/local/lib/python3.6/dist- 
packages/keras_preprocessing/image/directory_iterator.py in __init__(self, 
directory, image_data_generator, target_size, color_mode, classes, 
class_mode, batch_size, shuffle, seed, data_format, save_to_dir, 
save_prefix, save_format, follow_links, subset, interpolation, dtype)
    104         if not classes:
    105             classes = []
--> 106             for subdir in sorted(os.listdir(directory)):
    107                 if os.path.isdir(os.path.join(directory, subdir)):
    108                     classes.append(subdir)

FileNotFoundError: [Errno 2] No such file or directory: array([], 
dtype=float32)

Can someone help me? Thanks in advance.

Answer 1

What is your Model you want to fit? If it is a Tensorflow Model i would recomend tf.data, there you can simply build a dataset with:

import tensorflow as tf

IMAGEWIDTH = 100
IMAGEHEIGHT = 100
CHANNEL = 3
EPOCHS = 10

def get_label(file_path, class_names):
  # convert the path to a list of path components
  parts = tf.strings.split(file_path, os.path.sep)
  # The second to last is the class-directory
  return parts[-2] == class_names

def parse_image(filename):
    parts = tf.strings.split(filename, "\\")
    label = get_label(filename, CLASS_NAMES)
    
    image = tf.io.read_file(filename)
    image = tf.image.decode_png(image, channels=3)
    image = tf.image.convert_image_dtype(image, tf.float32)
    image = tf.image.resize(image, [IMAGEHEIGHT, IMAGEWIDTH])/255.0 # size the image and normalize
    return image, label

def make_dataset_unbatched():
    images_ds = list_ds.map(parse_image, num_parallel_calls=AUTOTUNE)
    images_ds = images_ds.repeat(EPOCHS)
    
    return images_ds

datasetFilePath = "/content/drive/My Drive/Colab Notebooks/Train/"
datasetPath = pathlib.Path(datasetFilePath)
list_ds = tf.data.Dataset.list_files(str(datasetPath/"*/*"))
num_elements = tf.data.experimental.cardinality(list_ds).numpy() # get the size of your dataset
CLASS_NAMES = np.array([item.name for item in datasetPath.glob('*')])

dataset = make_dataset_unbatched().batch(BATCH_SIZE, drop_remainder=True)
train_datagen.fit(dataset)

there you can also add multiple other tweaks to your dataset. For more information Tensorflow Dataset

I know its defenetly not the best code, but it might be a starting help. If its too bad, feel free to edit it.

Answer 2

You encounter a FileNotFoundError, which means that a file can not be found. I suspect that you forgot the filename extension of your image. Should '/content/drive/My Drive/Colab Notebooks/Train/Haze' not be something like '/content/drive/My Drive/Colab Notebooks/Train/Haze.png' or '/content/drive/My Drive/Colab Notebooks/Train/Haze.jpg'