memory error while converting images into an array

Question

I am working on a facial recognition use case. I have 57k jpg images and am converting them into an array. While executing the program, I am getting a memory error.

The function I am using:

def image_array(l):
    features = []
    for pgm in l:
        pic = image.load_img(pgm, target_size=(224, 224))
        x = image.img_to_array(pic)
        x = np.expand_dims(x, axis=0)
        x = preprocess_input(x)
        features.append(x)
    npfeatures = np.array(features)
    img_dt = np.rollaxis(npfeatures, 1, 0)
    return img_dt[0]

The input for this function is a list which looks like:

l =['/home/user/image1.jpg','/home/user/image2.jpg','/home/user/image3.jpg'......]

The error that I am getting:

Traceback (most recent call last):
  File "FR.py", line 145, in <module>
    vec_image1 = image_array(final_df['image1'].values.tolist())
  File "FR.py", line 140, in image_array
    npfeatures = np.array(features)
MemoryError

The imports that I used for above function are:

> from keras.preprocessing import image 
> from keras.applications.vgg16 import preprocess_input

Answer 1

You can actually compute how much memory it will take to hold 57,000 images in memory (it is a lot!). You are also holding them twice: once in the features list and then trying again in npfeatures. This second part will make a full copy of the entire features list. Hence why it runs out of memory there.

Here are some starting steps that should help you understand the limitations of your approach and perhaps get a working method:

1. You don't need this line: x = np.expand_dims(x, axis=0):

They do that in the Keras documentation, because you need a single image to have an additional dimension for the batch size. You do not need that because you append many images to a list (features), and the length of that list is the same thing, the batch size (the number of images).

2. Do your own scaling

In the case f VGG16 (and 19 I believe), the preprocess_input method simply scales pixel values between -1 and +1. You can probably do this a little more light-weight yourself. You can do the scaling on a numpy array like this:

x = (x / 127.5 - 1)

... and now remove the line with preprocess_input

3. Break it down into smaller pieces:

You can then either feed them directly into you models (if that is possible), or save the npfeatures to disk, one chunk at a time.

4. Try seeing how much memory your machine has while running this script:

Add a simple line after features.append(x), like this:

print("Loaded {} images"format(len(features)))

If you are running this on a Linux machine or Mac OSX, try using a tool like htop in terminal. There is also system monitor... same for Windows. You should be able to see the memory consumption grow until the point your script crashes. Now you know how many images you can do in one cycle - the last printed number before the crash.