1
$\begingroup$

I am trying to do image classificaition with a dataset that contains images of different sizes. The images are in a folder called Train, which contains 4 subfolders callsed HAZE,RAINY,SNOWY and SUNNY. I want to rescale all the images contained in these 4 subfolders. To do this, I use the following code:

from PIL import Image                                              
import os, sys                       

path = "/content/drive/My Drive/Colab Notebooks/Train"
dirs = os.listdir( path )                                       

def resize():
    for item in dirs:
        if os.path.isfile(path+item):
            im = Image.open(path+item)
            f, e = os.path.splitext(path+item)
            imResize = im.resize((254,254), Image.ANTIALIAS)
            imResize.save(f+'.png', 'png', quality=80)

resize()

the problem is that this piece of code doesn't do anything. I don't understand because it seems correct to me. The directory is to a google drive folder.

Can somebody please help me? Thanks in advance.

[EDIT]I have tried using the final slash:

path = "/content/drive/My Drive/Colab Notebooks/Train/"
dirs = os.listdir( path )

def resize():
for item in dirs:
    if os.path.isfile(path+item):
        im = Image.open(path+item)
        f, e = os.path.splitext(path+item)
        imResize = im.resize((200,200), Image.ANTIALIAS)
        imResize.save(f + ' resized.jpg', 'JPEG', quality=90)

but it tells me:

     OSError                                   Traceback (most recent call last)
<ipython-input-134-09ef2013485c> in <module>()
     13             imResize.save(f + ' resized.jpg', 'JPEG', quality=90)
     14 
---> 15 resize()

1 frames
/usr/local/lib/python3.6/dist-packages/PIL/Image.py in open(fp, mode)
   2570         fp.close()
   2571     raise IOError("cannot identify image file %r"
-> 2572                   % (filename if filename else fp))
   2573 
   2574 #

OSError: cannot identify image file '/content/drive/My Drive/Colab 
Notebooks/Train/Untitled1.ipynb'

[EDIT]I have tried to create a new folder with few sample imaged, I called it prova, and it works:

path = '/content/drive/My Drive/prova/'
dirs = os.listdir( path )

def resize():
    for item in dirs:
        print('entered')
        if item.endswith('.jpg'):
            im = Image.open(path+item)
            f, e = os.path.splitext(path+item)
            imResize = im.resize((200,200), Image.ANTIALIAS)
            imResize.save(f + ' resized.jpg', 'JPEG', quality=90)
            print('done')

resize()

and it works for this sample code. So I think the problem is how to access the sub-folders.

[EDIT]Now that the images are resized, I use a generator:

trainingset = '/content/drive/My Drive/Colab Notebooks/Train'
testset = '/content/drive/My Drive/Colab Notebooks/Test'


batch_size = 32
train_datagen = ImageDataGenerator(
    featurewise_center=True,
    featurewise_std_normalization=True,
    #rescale = 1. / 255,
    \
    zoom_range=0.1,\
    rotation_range=10,\
    width_shift_range=0.1,\
    height_shift_range=0.1,\
    horizontal_flip=True,\
    vertical_flip=False)



train_generator = train_datagen.flow_from_directory(
    directory=trainingset,
    #target_size=(256, 256),
    color_mode="rgb",
    batch_size=batch_size,
    class_mode="categorical",
    shuffle=True
)


test_datagen = ImageDataGenerator(
    featurewise_center=True,
    featurewise_std_normalization=True,
    rescale = 1. / 255
   )



test_generator = test_datagen.flow_from_directory(
    directory=testset,
    #target_size=(256, 256),
    color_mode="rgb",
    batch_size=batch_size,
    class_mode="categorical",
    shuffle=False
)

num_samples = train_generator.n
num_classes = train_generator.num_classes
input_shape = train_generator.image_shape

classnames = [k for k,v in train_generator.class_indices.items()]

I have commented some fields inside the Imagegenerator such as target_size=(256, 256) and rescale = 1. / 255 since they rescale the image again. But after I ran this code, I try to print the images to see if they are still scaled:

import matplotlib.pyplot as plt

n = 10
x,y = train_generator.next()
# x,y size is train_generator.batch_size

for i in range(0,n):
    image = x[i]
    label = y[i].argmax()  # categorical from one-hot-encoding
    print(classnames[label])
    plt.imshow(image)
    plt.show()

but they are all like these:

enter image description here

and also they are not scaled.

Why does this happens?

$\endgroup$
  • $\begingroup$ is it working on local dir? $\endgroup$ – Ilker Kurtulus Nov 28 '19 at 16:06
  • 1
    $\begingroup$ Put a "/content/drive/My Drive/Colab Notebooks/Train/" slash after train because you are accessing the subfolders present in your directory and run this loop 4 times because when you use "+" for string concatenation it doesn't add that slash and so use it somewhat like this: r"/content/drive/My Drive/Colab Notebooks/Train/" this should work. $\endgroup$ – khwaja wisal Dec 10 '19 at 19:14
  • $\begingroup$ Thanks for answering. By adding the final slash something happens, but it gives me an error. I have edited the question with more details. $\endgroup$ – J.D. Dec 10 '19 at 19:32
0
+50
$\begingroup$

I would suggest for you to use the walk() function from the os package for iterating over all your (sub-)directories and files.

Below is a sample code:

import os

main_directory = "/content/drive/My Drive/Colab Notebooks/Train/"


for subdir, dirs, files in os.walk(main_directory):
    for file in files:
        filepath = os.path.join(subdir, file)

        if filepath.endswith(".jpg"):
            im = Image.open(filepath)
            imResize = im.resize((200, 200), Image.ANTIALIAS)
            imResize.save(filepath[:-4] + '_resized.jpg', 'JPEG', quality=90)

Pro tip: if you want to see the progress of iterating over the different subdirectories, you can use the tqdm package.

import os

from tqdm import tqdm # pip install tqdm

main_directory = "/content/drive/My Drive/Colab Notebooks/Train/"


for subdir, dirs, files in os.walk(main_directory):
    for file in tqdm(files, desc='Loading files from {}'.format(subdir)):
        filepath = os.path.join(subdir, file)

        if filepath.endswith(".jpg"):
            im = Image.open(filepath)
            imResize = im.resize((200, 200), Image.ANTIALIAS)
            imResize.save(filepath[:-4] + '_resized.jpg', 'JPEG', quality=90)
$\endgroup$
  • $\begingroup$ Thanks for answering. I tried your code, but the images don't get scaled $\endgroup$ – J.D. Dec 11 '19 at 5:09
  • $\begingroup$ @J.D. What do you mean the images don't get scaled? 1) is the new image being created (with the `_resized) extension? 2) was was the size of the original image? $\endgroup$ – BrunoGL Dec 11 '19 at 6:48
  • $\begingroup$ sorry, ignore my previous comment. What happens is that it gives me an error : OSError: cannot write mode RGBA as JPEG $\endgroup$ – J.D. Dec 11 '19 at 8:40
  • $\begingroup$ the orginal images are all of different sizes. What I want to do is take all these images and give to all of them the same size. $\endgroup$ – J.D. Dec 11 '19 at 10:36
  • $\begingroup$ did you see this? github.com/python-pillow/Pillow/issues/2609 Seems like you cannot save as JPEG. Is saving as PNG ok for you? $\endgroup$ – BrunoGL Dec 11 '19 at 10:46
0
$\begingroup$

When you write path+item, you are not getting a valid path to a file. This is because there is no separator at the end of path. For example, if img1.png is in the directory, path+item gives “/content/drive/My Drive/Colab Notebooks/Trainimg1.png”.

Also, it seems you are iterating through the subdirectories but not the files within those directories.

A better way to do this is to write os.path.join(path, dir, file) where file is the result of iterating through dir.

$\endgroup$
0
$\begingroup$

The OSError is caused by the fact that you have a .ipynb file in the directory which is not an image, which therefore can't be opened using Image.open. You should probably change your if statement so it only opens files of a specific type.

path = "/content/drive/My Drive/Colab Notebooks/Train/"
dirs = os.listdir( path )

def resize():
    for item in dirs:
        if os.path.isfile(path+item) and item.endswith('.png'):
            im = Image.open(path+item)
            f, e = os.path.splitext(path+item)
            imResize = im.resize((200,200), Image.ANTIALIAS)
            imResize.save(f + ' resized.jpg', 'JPEG', quality=90)

This is assuming your image files all have a .png file extension. If your image files have a different extension (such as .jpg) you can simply change the extension.

$\endgroup$
  • $\begingroup$ Thanks for the answer. It does not work by doing so. It doesn' t do anything. $\endgroup$ – J.D. Dec 10 '19 at 19:46
  • $\begingroup$ Have you tried adding a print statement within the loop just to see whether the loop was entered or not? $\endgroup$ – Oxbowerce Dec 10 '19 at 20:07
  • $\begingroup$ yes, it enters the for loop, but it does not enter the if condition. $\endgroup$ – J.D. Dec 10 '19 at 20:09
  • $\begingroup$ What types of images are you trying to read? Can you perhaps share a google colab/notebook link with your code in it? $\endgroup$ – Oxbowerce Dec 10 '19 at 20:11
  • $\begingroup$ The images are in .jpg format and represent types of weather conditions. I am able to scale them correctly by using an Image generator, but by doing so I get a warning for which I asked a question which I attach here to you datascience.stackexchange.com/questions/64402/… . So to overcome this problem My idea is to scale first the images, and then use a generator and train the network. This is why I am trying to scale first, but with a generator was much easier. $\endgroup$ – J.D. Dec 10 '19 at 20:22
0
$\begingroup$

If you are using Python 3.6 (or higher), pathlib cleans up the code by automatically walking all subdirectories and processing only images:

from pathlib import Path

from PIL import Image  

path = Path('/content/drive/My Drive/prova/')

for filename in path.glob('**/*.jpg'):
    im = Image.open(filename)
    im_resized = im.resize((200,200), Image.ANTIALIAS)
    im_resized.save(filename.parent / Path(filename.stem + '_resized.jpg'), 'JPEG', quality=90)

$\endgroup$
  • $\begingroup$ Thanks, but it does not scale the images $\endgroup$ – J.D. Dec 11 '19 at 5:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.