scaling images for image classification

Question

I am trying to do image classificaition with a dataset that contains images of different sizes. The images are in a folder called Train, which contains 4 subfolders callsed HAZE,RAINY,SNOWY and SUNNY. I want to rescale all the images contained in these 4 subfolders. To do this, I use the following code:

from PIL import Image                                              
import os, sys                       

path = "/content/drive/My Drive/Colab Notebooks/Train"
dirs = os.listdir( path )                                       

def resize():
    for item in dirs:
        if os.path.isfile(path+item):
            im = Image.open(path+item)
            f, e = os.path.splitext(path+item)
            imResize = im.resize((254,254), Image.ANTIALIAS)
            imResize.save(f+'.png', 'png', quality=80)

resize()

the problem is that this piece of code doesn't do anything. I don't understand because it seems correct to me. The directory is to a google drive folder.

Can somebody please help me? Thanks in advance.

[EDIT]I have tried using the final slash:

path = "/content/drive/My Drive/Colab Notebooks/Train/"
dirs = os.listdir( path )

def resize():
for item in dirs:
    if os.path.isfile(path+item):
        im = Image.open(path+item)
        f, e = os.path.splitext(path+item)
        imResize = im.resize((200,200), Image.ANTIALIAS)
        imResize.save(f + ' resized.jpg', 'JPEG', quality=90)

but it tells me:

     OSError                                   Traceback (most recent call last)
<ipython-input-134-09ef2013485c> in <module>()
     13             imResize.save(f + ' resized.jpg', 'JPEG', quality=90)
     14 
---> 15 resize()

1 frames
/usr/local/lib/python3.6/dist-packages/PIL/Image.py in open(fp, mode)
   2570         fp.close()
   2571     raise IOError("cannot identify image file %r"
-> 2572                   % (filename if filename else fp))
   2573 
   2574 #

OSError: cannot identify image file '/content/drive/My Drive/Colab 
Notebooks/Train/Untitled1.ipynb'

[EDIT]I have tried to create a new folder with few sample imaged, I called it prova, and it works:

path = '/content/drive/My Drive/prova/'
dirs = os.listdir( path )

def resize():
    for item in dirs:
        print('entered')
        if item.endswith('.jpg'):
            im = Image.open(path+item)
            f, e = os.path.splitext(path+item)
            imResize = im.resize((200,200), Image.ANTIALIAS)
            imResize.save(f + ' resized.jpg', 'JPEG', quality=90)
            print('done')

resize()

and it works for this sample code. So I think the problem is how to access the sub-folders.

[EDIT]Now that the images are resized, I use a generator:

trainingset = '/content/drive/My Drive/Colab Notebooks/Train'
testset = '/content/drive/My Drive/Colab Notebooks/Test'


batch_size = 32
train_datagen = ImageDataGenerator(
    featurewise_center=True,
    featurewise_std_normalization=True,
    #rescale = 1. / 255,
    \
    zoom_range=0.1,\
    rotation_range=10,\
    width_shift_range=0.1,\
    height_shift_range=0.1,\
    horizontal_flip=True,\
    vertical_flip=False)



train_generator = train_datagen.flow_from_directory(
    directory=trainingset,
    #target_size=(256, 256),
    color_mode="rgb",
    batch_size=batch_size,
    class_mode="categorical",
    shuffle=True
)


test_datagen = ImageDataGenerator(
    featurewise_center=True,
    featurewise_std_normalization=True,
    rescale = 1. / 255
   )



test_generator = test_datagen.flow_from_directory(
    directory=testset,
    #target_size=(256, 256),
    color_mode="rgb",
    batch_size=batch_size,
    class_mode="categorical",
    shuffle=False
)

num_samples = train_generator.n
num_classes = train_generator.num_classes
input_shape = train_generator.image_shape

classnames = [k for k,v in train_generator.class_indices.items()]

I have commented some fields inside the Imagegenerator such as target_size=(256, 256) and rescale = 1. / 255 since they rescale the image again. But after I ran this code, I try to print the images to see if they are still scaled:

import matplotlib.pyplot as plt

n = 10
x,y = train_generator.next()
# x,y size is train_generator.batch_size

for i in range(0,n):
    image = x[i]
    label = y[i].argmax()  # categorical from one-hot-encoding
    print(classnames[label])
    plt.imshow(image)
    plt.show()

but they are all like these:

and also they are not scaled.

Why does this happens?

Answer 1

I would suggest for you to use the walk() function from the os package for iterating over all your (sub-)directories and files.

Below is a sample code:

import os

main_directory = "/content/drive/My Drive/Colab Notebooks/Train/"


for subdir, dirs, files in os.walk(main_directory):
    for file in files:
        filepath = os.path.join(subdir, file)

        if filepath.endswith(".jpg"):
            im = Image.open(filepath)
            imResize = im.resize((200, 200), Image.ANTIALIAS)
            imResize.save(filepath[:-4] + '_resized.jpg', 'JPEG', quality=90)

Pro tip: if you want to see the progress of iterating over the different subdirectories, you can use the tqdm package.

import os

from tqdm import tqdm # pip install tqdm

main_directory = "/content/drive/My Drive/Colab Notebooks/Train/"


for subdir, dirs, files in os.walk(main_directory):
    for file in tqdm(files, desc='Loading files from {}'.format(subdir)):
        filepath = os.path.join(subdir, file)

        if filepath.endswith(".jpg"):
            im = Image.open(filepath)
            imResize = im.resize((200, 200), Image.ANTIALIAS)
            imResize.save(filepath[:-4] + '_resized.jpg', 'JPEG', quality=90)

Answer 2

When you write path+item, you are not getting a valid path to a file. This is because there is no separator at the end of path. For example, if img1.png is in the directory, path+item gives “/content/drive/My Drive/Colab Notebooks/Trainimg1.png”.

Also, it seems you are iterating through the subdirectories but not the files within those directories.

A better way to do this is to write os.path.join(path, dir, file) where file is the result of iterating through dir.

Answer 3

The OSError is caused by the fact that you have a .ipynb file in the directory which is not an image, which therefore can't be opened using Image.open. You should probably change your if statement so it only opens files of a specific type.

path = "/content/drive/My Drive/Colab Notebooks/Train/"
dirs = os.listdir( path )

def resize():
    for item in dirs:
        if os.path.isfile(path+item) and item.endswith('.png'):
            im = Image.open(path+item)
            f, e = os.path.splitext(path+item)
            imResize = im.resize((200,200), Image.ANTIALIAS)
            imResize.save(f + ' resized.jpg', 'JPEG', quality=90)

This is assuming your image files all have a .png file extension. If your image files have a different extension (such as .jpg) you can simply change the extension.

Answer 4

If you are using Python 3.6 (or higher), pathlib cleans up the code by automatically walking all subdirectories and processing only images:

from pathlib import Path

from PIL import Image  

path = Path('/content/drive/My Drive/prova/')

for filename in path.glob('**/*.jpg'):
    im = Image.open(filename)
    im_resized = im.resize((200,200), Image.ANTIALIAS)
    im_resized.save(filename.parent / Path(filename.stem + '_resized.jpg'), 'JPEG', quality=90)