# How to increase the accuracy of my predictions (CNN fine tuning VGG16 KERAS)

In my VGG16 fine-tuning, I have to classify retinal images in 2 classes (or 4th stage or not 4th stage) and I have 700 images each class to train. This is my code now:


TRAIN_DIR = 'train/'
TEST_DIR = 'test/'
v = 'v/'
BATCH_SIZE = 32
NUM_EPOCHS = 5

LabelList = list()
ImageCV = list()
classes = ["nonPdr", "pdr"]

# Get all subdirectories
FolderList = [f for f in os.listdir(Path) if not f.startswith('.')]

# Loop over each directory
for File in FolderList:
for index, Image in enumerate(os.listdir(os.path.join(Path, File))):
# Convert the path into a file
ImageCV.append(cv2.resize(cv2.imread(os.path.join(Path, File) + os.path.sep + Image), (224,224)))
#ImageCV[index]= np.array(ImageCV[index]) / 255.0
LabelList.append(classes.index(os.path.splitext(File)[0]))

#ImageCV[index] = cv2.addWeighted(ImageCV[index],4, cv2.GaussianBlur(ImageCV[index],(0,0), 10), -4, 128)
image_blurred = cv2.GaussianBlur(ImageCV[index], (0, 0), 100 / 30)
ImageCV[index] = cv2.addWeighted(ImageCV[index], 4, image_blurred, -4, 128)

return ImageCV, LabelList

vgg16_model = VGG16(weights="imagenet", include_top=True)

# (1) visualize layers
print("VGG16 model layers")
for i, layer in enumerate(vgg16_model.layers):
print(i, layer.name, layer.output_shape)

# (2) remove the top layer
base_model = Model(input=vgg16_model.input,
output=vgg16_model.get_layer("block5_pool").output)

# (3) attach a new top layer
base_out = base_model.output
base_out = Reshape((25088,))(base_out)
top_fc1 = Dropout(0.5)(base_out)
# output layer: (None, 5)
top_preds = Dense(1, activation="sigmoid")(top_fc1)

# (4) freeze weights until the last but one convolution layer (block4_pool)
for layer in base_model.layers[0:14]:
layer.trainable = False

# (5) create new hybrid model
model = Model(input=base_model.input, output=top_preds)

# (6) compile and train the model
sgd = SGD(lr=1e-4, momentum=0.9)
model.compile(optimizer=sgd, loss="binary_crossentropy", metrics=["accuracy"])

datagen = ImageDataGenerator(
featurewise_center=True,
featurewise_std_normalization=True,
rotation_range=20,
width_shift_range=0.2,
height_shift_range=0.2,
horizontal_flip=True)

# compute quantities required for featurewise normalization
# (std, mean, and principal components if ZCA whitening is applied)
datagen.fit(data)
es = EarlyStopping(monitor='val_loss', verbose=1)

# fits the model on batches with real-time data augmentation:
model.fit_generator(datagen.flow(np.array(data), np.array(labels), batch_size=32),
steps_per_epoch=len(np.array(data)) / 32, epochs=15,
validation_data=(np.array(valid), np.array(vlabels)),
nb_val_samples=72, callbacks=[es])

model.save('model.h5')


This model, in 4 epochs, gets a accuracy of 1.00 with a minimal loss (~0.01), but when I try to run the predict.py in some test images, the REAL accuracy is about 50% (in 10 images, the app answers correctly only 5). This is my predict. py:

model = load_model('model.h5')

for filename in os.listdir(r'v/'):
if filename.endswith(".jpg") or filename.endswith(".ppm") or filename.endswith(".jpeg") or filename.endswith(".png"):
ImageCV = cv2.resize(cv2.imread(os.path.join(TEST_DIR) + filename), (224,224))
#ImageCV = cv2.addWeighted(ImageCV,4, cv2.GaussianBlur(ImageCV,(0,0), 224/25), -4, 120)
#ImageCV = ImageCV.reshape(-1,224,224,3)

x = image.img_to_array(ImageCV)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
#print(model.predict(x))

prob = model.predict(x)
if prob < 0.5:
print("nonPDR")
else:
print("PDR")
print(filename)


THINGS I'VE TRIED:
- change LR
- put dense(256) layer before dropout
- increase dropout to 0.75
- change some dataAugmentation parameters

The input will be images like this : https://i.imgur.com/DsU06Xv.jpg To be better to classificate, i'm refining my image in ReadImages method: https://i.imgur.com/YBzaBgw.png

So, what can I do to improve my accuracy REALLY in my predictions and not only in the train and accuracy values?

There's something I've been making wrong?

## UPDATE

I've drop out the gaussianblur I was using in ReadImages, and included the follow:

data = np.asarray(data)
valid = np.asarray(valid)

data = data.astype('float32')
valid = valid.astype('float32')

data /= 255
valid /= 255


And after run my train.py:

Epoch 1/15

44/43 [==============================] - 476s 11s/step - loss: 0.7153 - acc: 0.5788 - val_loss: 0.6937 - val_acc: 0.5556

Epoch 2/15

44/43 [==============================] - 468s 11s/step - loss: 0.5526 - acc: 0.7275 - val_loss: 0.6838 - val_acc: 0.5833

Epoch 3/15

44/43 [==============================] - 474s 11s/step - loss: 0.5068 - acc: 0.7595 - val_loss: 0.6927 - val_acc: 0.5694

Epoch 00003: early stopping


After, I update the std and mean on predict.py:

for filename in os.listdir(r'v/'):
if filename.endswith(".jpg") or filename.endswith(".ppm") or filename.endswith(".jpeg") or filename.endswith(".png"):
ImageCV = cv2.resize(cv2.imread(os.path.join(TEST_DIR) + filename), (224,224))

ImageCV = np.asarray(ImageCV)

ImageCV = ImageCV.astype('float32')

ImageCV /= 255
x = ImageCV

x = np.expand_dims(x, axis=0)
x = normalize(x, [0.12810835, 0.17897758, 0.23883381], [0.14304605, 0.18229756, 0.2362126])

prob = model.predict(x)
if prob <= 0.50:
print("nonPDR >>>", filename)
nonPdr += 1
else:
print("PDR >>>", filename)
pdr += 1
print(prob)
print("Number of retinas with PDR: ",pdr)
print("Number of retinas without PDR: ",nonPdr)


And after run this code, I'm getting roughly 75% accuracy in my test dir..

So, can I improve something, or this is the maximum for these tiny number of images?

I have the following suggestions.

1. Make sure your preprocess function in predict.py is doing exactly what datagen is doing during training (featurewise_center and normalization)
2. For Sigmoid value greater than 0.5 is defined as class 1 and value less than or equal is defined as class 0. In your code, for values exactly same as 0.5 are being assigned in class 1 which should not be the case.

For point 2, you may argue why this equality will make a difference. Most of the time in sigmoid due to some reason the sigmoid value for class 0 comes as exact 0.5 which especially I have seen in segmentation tasks.

Edit: To ensure that image is processed in same way as the training images are processed use keras generator only

test_datagen = ImageDataGenerator(samplewise_center=True,
samplewise_std_normalization=True)

test_generator = test_datagen.flow_from_directory(
args.validate_dir,  # this is the target directory
target_size=(512, 512),
batch_size=batch_size,
class_mode='categorical', shuffle=True)


Now test_generator.next() will return X,Y pair which can be used for prediction checks.

I have ignored rotations and all from ImageDataGenerator as they are not changing the image properties like featurewise_center is changing.

• How can I make in predict the same as in train? Oct 3 '19 at 12:39
• Check the answer now. Oct 3 '19 at 12:58
• Great answer, but i'm not understanding how to apply it in my project.. The code you've posted is on predict.py? I have to write my gaussian blur too? Oct 3 '19 at 13:47
• Use ReadImage method in predict.py as well. Also yes, the code I have posted is for predict.py. Hope I am being helpful. Oct 3 '19 at 14:20
• see my update, please Oct 7 '19 at 12:48