1
$\begingroup$

In my VGG16 fine-tuning, I have to classify retinal images in 2 classes (or 4th stage or not 4th stage) and I have 700 images each class to train. This is my code now:


TRAIN_DIR = 'train/'
TEST_DIR = 'test/'
v = 'v/'
BATCH_SIZE = 32
NUM_EPOCHS = 5

def ReadImages(Path):
    LabelList = list()
    ImageCV = list()
    classes = ["nonPdr", "pdr"]

    # Get all subdirectories
    FolderList = [f for f in os.listdir(Path) if not f.startswith('.')]

    # Loop over each directory
    for File in FolderList:
        for index, Image in enumerate(os.listdir(os.path.join(Path, File))):
            # Convert the path into a file
            ImageCV.append(cv2.resize(cv2.imread(os.path.join(Path, File) + os.path.sep + Image), (224,224)))
            #ImageCV[index]= np.array(ImageCV[index]) / 255.0
            LabelList.append(classes.index(os.path.splitext(File)[0])) 

            #ImageCV[index] = cv2.addWeighted(ImageCV[index],4, cv2.GaussianBlur(ImageCV[index],(0,0), 10), -4, 128)
            image_blurred = cv2.GaussianBlur(ImageCV[index], (0, 0), 100 / 30)
            ImageCV[index] = cv2.addWeighted(ImageCV[index], 4, image_blurred, -4, 128)



    return ImageCV, LabelList

data, labels = ReadImages(TRAIN_DIR)
valid, vlabels = ReadImages(TEST_DIR)

vgg16_model = VGG16(weights="imagenet", include_top=True)

# (1) visualize layers
print("VGG16 model layers")
for i, layer in enumerate(vgg16_model.layers):
    print(i, layer.name, layer.output_shape)

# (2) remove the top layer
base_model = Model(input=vgg16_model.input, 
                   output=vgg16_model.get_layer("block5_pool").output)

# (3) attach a new top layer
base_out = base_model.output
base_out = Reshape((25088,))(base_out)
top_fc1 = Dropout(0.5)(base_out)
# output layer: (None, 5)
top_preds = Dense(1, activation="sigmoid")(top_fc1)

# (4) freeze weights until the last but one convolution layer (block4_pool)
for layer in base_model.layers[0:14]:
    layer.trainable = False

# (5) create new hybrid model
model = Model(input=base_model.input, output=top_preds)

# (6) compile and train the model
sgd = SGD(lr=1e-4, momentum=0.9)
model.compile(optimizer=sgd, loss="binary_crossentropy", metrics=["accuracy"])

datagen = ImageDataGenerator(
    featurewise_center=True,
    featurewise_std_normalization=True,
    rotation_range=20,
    width_shift_range=0.2,
    height_shift_range=0.2,
    horizontal_flip=True)

# compute quantities required for featurewise normalization
# (std, mean, and principal components if ZCA whitening is applied)
datagen.fit(data)
es = EarlyStopping(monitor='val_loss', verbose=1)

# fits the model on batches with real-time data augmentation:
model.fit_generator(datagen.flow(np.array(data), np.array(labels), batch_size=32), 
                    steps_per_epoch=len(np.array(data)) / 32, epochs=15,
                    validation_data=(np.array(valid), np.array(vlabels)),
                    nb_val_samples=72, callbacks=[es])


model.save('model.h5')

This model, in 4 epochs, gets a accuracy of 1.00 with a minimal loss (~0.01), but when I try to run the predict.py in some test images, the REAL accuracy is about 50% (in 10 images, the app answers correctly only 5). This is my predict. py:

model = load_model('model.h5')

for filename in os.listdir(r'v/'):
    if filename.endswith(".jpg") or filename.endswith(".ppm") or filename.endswith(".jpeg") or filename.endswith(".png"):
        ImageCV = cv2.resize(cv2.imread(os.path.join(TEST_DIR) + filename), (224,224))
        #ImageCV = cv2.addWeighted(ImageCV,4, cv2.GaussianBlur(ImageCV,(0,0), 224/25), -4, 120)
        #ImageCV = ImageCV.reshape(-1,224,224,3)

        x = image.img_to_array(ImageCV)
        x = np.expand_dims(x, axis=0)
        x = preprocess_input(x)
        #print(model.predict(x))

        prob = model.predict(x)
        if prob < 0.5:
            print("nonPDR")
        else:
            print("PDR")
        print(filename)

THINGS I'VE TRIED:
- change LR
- put dense(256) layer before dropout
- increase dropout to 0.75
- change some dataAugmentation parameters

The input will be images like this : https://i.imgur.com/DsU06Xv.jpg To be better to classificate, i'm refining my image in ReadImages method: https://i.imgur.com/YBzaBgw.png

So, what can I do to improve my accuracy REALLY in my predictions and not only in the train and accuracy values?

There's something I've been making wrong?

UPDATE

I've drop out the gaussianblur I was using in ReadImages, and included the follow:

data = np.asarray(data)
valid = np.asarray(valid)

data = data.astype('float32')
valid = valid.astype('float32')

data /= 255
valid /= 255

And after run my train.py:

Epoch 1/15

44/43 [==============================] - 476s 11s/step - loss: 0.7153 - acc: 0.5788 - val_loss: 0.6937 - val_acc: 0.5556

Epoch 2/15

44/43 [==============================] - 468s 11s/step - loss: 0.5526 - acc: 0.7275 - val_loss: 0.6838 - val_acc: 0.5833

Epoch 3/15

44/43 [==============================] - 474s 11s/step - loss: 0.5068 - acc: 0.7595 - val_loss: 0.6927 - val_acc: 0.5694

Epoch 00003: early stopping

After, I update the std and mean on predict.py:

for filename in os.listdir(r'v/'):
    if filename.endswith(".jpg") or filename.endswith(".ppm") or filename.endswith(".jpeg") or filename.endswith(".png"):
        ImageCV = cv2.resize(cv2.imread(os.path.join(TEST_DIR) + filename), (224,224))

        ImageCV = np.asarray(ImageCV)

        ImageCV = ImageCV.astype('float32')

        ImageCV /= 255  
        x = ImageCV

        x = np.expand_dims(x, axis=0)
        x = normalize(x, [0.12810835, 0.17897758, 0.23883381], [0.14304605, 0.18229756, 0.2362126])

        prob = model.predict(x)
        if prob <= 0.50:
            print("nonPDR >>>", filename)
            nonPdr += 1
        else:
            print("PDR >>>", filename)
            pdr += 1
        print(prob)
print("Number of retinas with PDR: ",pdr)
print("Number of retinas without PDR: ",nonPdr)

And after run this code, I'm getting roughly 75% accuracy in my test dir..

So, can I improve something, or this is the maximum for these tiny number of images?

$\endgroup$
2
$\begingroup$

I have the following suggestions.

  1. Make sure your preprocess function in predict.py is doing exactly what datagen is doing during training (featurewise_center and normalization)
  2. For Sigmoid value greater than 0.5 is defined as class 1 and value less than or equal is defined as class 0. In your code, for values exactly same as 0.5 are being assigned in class 1 which should not be the case.

For point 2, you may argue why this equality will make a difference. Most of the time in sigmoid due to some reason the sigmoid value for class 0 comes as exact 0.5 which especially I have seen in segmentation tasks.

Edit: To ensure that image is processed in same way as the training images are processed use keras generator only

test_datagen = ImageDataGenerator(samplewise_center=True,
            samplewise_std_normalization=True)

test_generator = test_datagen.flow_from_directory(
        args.validate_dir,  # this is the target directory
        target_size=(512, 512),  
        batch_size=batch_size,
        class_mode='categorical', shuffle=True)

Now test_generator.next() will return X,Y pair which can be used for prediction checks.

I have ignored rotations and all from ImageDataGenerator as they are not changing the image properties like featurewise_center is changing.

$\endgroup$
  • 1
    $\begingroup$ How can I make in predict the same as in train? $\endgroup$ – Gilberto Oct 3 '19 at 12:39
  • $\begingroup$ Check the answer now. $\endgroup$ – shivam shah Oct 3 '19 at 12:58
  • $\begingroup$ Great answer, but i'm not understanding how to apply it in my project.. The code you've posted is on predict.py? I have to write my gaussian blur too? $\endgroup$ – Gilberto Oct 3 '19 at 13:47
  • $\begingroup$ Use ReadImage method in predict.py as well. Also yes, the code I have posted is for predict.py. Hope I am being helpful. $\endgroup$ – shivam shah Oct 3 '19 at 14:20
  • $\begingroup$ see my update, please $\endgroup$ – Gilberto Oct 7 '19 at 12:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.