0
$\begingroup$

First post here. I'm working on a project about multi-class image classification and created a python script using Keras to train a model with transfer learning. To my dismay the model has always predicted the same class, I've simplified the model down to 3 image classes (I'm using a kaggle food image stock with 800 training samples and 800 validation samples per class plus image reformatting) and tried different optimizers, yet it still comes down to the same class while the model also apparently only has an accuracy of ~0.2563 at 25 epochs of training. I've posted the code below, how can I improve the accuracy of this script and solve the same predicted class problem?

import pandas as pd
import numpy as np
import os
import keras
import matplotlib.pyplot as plt
from keras.layers import Dense, GlobalAveragePooling2D
from keras.preprocessing import image
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Model
from keras import optimizers
from keras import applications
from keras.applications.vgg16 import preprocess_input

img_classes = 3

base_model = applications.VGG16(weights='imagenet', include_top=False)

x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(1024, activation='relu')(x)
x = Dense(1024, activation='relu')(x)
x = Dense(512, activation='relu')(x)
preds = Dense(img_classes, activation='softmax')(x)

model = Model(inputs=base_model.input, outputs=preds)

for i, layer in enumerate(model.layers):
    print(i, layer.name)

for layer in model.layers[:25]:
    layer.trainable = False

train_datagen = ImageDataGenerator(rescale=1./255,
                                   rotation_range=40,
                                   width_shift_range=0.2,
                                   height_shift_range=0.2,
                                   shear_range=0.2,
                                   zoom_range=0.2,
                                   horizontal_flip=True,
                                   fill_mode='nearest',
                                   preprocessing_function=preprocess_input)

train_generator = train_datagen.flow_from_directory('./food-101/bigtrain',
                                                    target_size=(128, 128),
                                                    color_mode='rgb',
                                                    classes=['apple_pie', 'churros', 'miso_soup'],
                                                    batch_size=1,
                                                    class_mode='categorical',
                                                    shuffle=True)

val_datagen = ImageDataGenerator(
    rescale=1./255,
    rotation_range=40,
    width_shift_range=0.2,
    height_shift_range=0.2,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True,
    fill_mode='nearest',
    preprocessing_function=preprocess_input,)

val_generator = val_datagen.flow_from_directory(
    './food-101/bigval',
    target_size=(128, 128),
    classes=['apple_pie', 'churros', 'miso_soup'],
    batch_size=1,
    class_mode='categorical',
    shuffle=True)

# model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])

model.compile(optimizer=optimizers.SGD(lr=0.00001,
                                       momentum=0.9,
                                       decay=0.0001,
                                       nesterov=True), loss='categorical_crossentropy', metrics=['accuracy'])

batch_size = 1


validation_steps = 64 // batch_size
step_size_train = train_generator.n//train_generator.batch_size

model.fit_generator(generator=train_generator,
                    steps_per_epoch=step_size_train,
                    epochs=25,
                    validation_data=val_generator,
                    validation_steps=validation_steps)

model.save('./test_try_vgg_9.h5')

Example prediction results:

classes: apple_pie, churros, miso_soup

miso soup
[0.3202575  0.48074356 0.19899891] rmsprop 
[0.45246536 0.4505403  0.09699439] sgd

churros
[0.37473327 0.35784692 0.2674198 ] rmsprop
[0.4145825  0.465228   0.12018944] sgd

This is the prediction script:

from keras.models import load_model
from keras import optimizers
from keras.preprocessing import image
import numpy as np
from keras.applications.vgg16 import preprocess_input

# dimensions of our images
img_width, img_height = 512, 512

# load model
model = load_model('./test_try_vgg_9.h5')

# predicting images
img = image.load_img('./food-101/training/apple_pie/551535.jpg')
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)

pred = model.predict(x)
print("Probability: ")
print(pred[0])
$\endgroup$
1
  • $\begingroup$ Are you sure the 25 that you're setting to untrainable matches with the number of layers of vgg16? Otherwise, maybe try smaller network on top of vgg16. Also, batch size of 1 is kind of small and will make your training slower, so you could increase that and train for longer than 25 epochs $\endgroup$ – Shadi Jan 26 '19 at 7:59
2
$\begingroup$

You have made all the model's parameters untrainable.

You can easily check it by printing out a summary of the model:

model.summary() # This command prints the summary of the model

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, None, None, 3)     0         
_________________________________________________________________
block1_conv1 (Conv2D)        (None, None, None, 64)    1792      
_________________________________________________________________
block1_conv2 (Conv2D)        (None, None, None, 64)    36928     
_________________________________________________________________
block1_pool (MaxPooling2D)   (None, None, None, 64)    0         
_________________________________________________________________
block2_conv1 (Conv2D)        (None, None, None, 128)   73856     
_________________________________________________________________
block2_conv2 (Conv2D)        (None, None, None, 128)   147584    
_________________________________________________________________
block2_pool (MaxPooling2D)   (None, None, None, 128)   0         
_________________________________________________________________
block3_conv1 (Conv2D)        (None, None, None, 256)   295168    
_________________________________________________________________
block3_conv2 (Conv2D)        (None, None, None, 256)   590080    
_________________________________________________________________
block3_conv3 (Conv2D)        (None, None, None, 256)   590080    
_________________________________________________________________
block3_pool (MaxPooling2D)   (None, None, None, 256)   0         
_________________________________________________________________
block4_conv1 (Conv2D)        (None, None, None, 512)   1180160   
_________________________________________________________________
block4_conv2 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block4_conv3 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block4_pool (MaxPooling2D)   (None, None, None, 512)   0         
_________________________________________________________________
block5_conv1 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block5_conv2 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block5_conv3 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block5_pool (MaxPooling2D)   (None, None, None, 512)   0         
_________________________________________________________________
global_average_pooling2d_1 ( (None, 512)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 1024)              525312    
_________________________________________________________________
dense_2 (Dense)              (None, 1024)              1049600   
_________________________________________________________________
dense_3 (Dense)              (None, 512)               524800    
_________________________________________________________________
dense_4 (Dense)              (None, 3)                 1539      
=================================================================
Total params: 16,815,939
Trainable params: 0
Non-trainable params: 16,815,939
_________________________________________________________________

As you can see at the bottom of the summary, you have 0 trainable weights. To get what you wanted, change the for loop of where you set the layers trainable parameter from [:25] to [:20]:

for layer in model.layers[:20]:
    layer.trainable = False

model.layers[20] is your first new layer (named 'dense_1' in the summary):

model.layers[20].name
'dense_1'
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.