1
$\begingroup$

In my project I have 700 images for each class (pdr and nonPdr) totalizing 1400 images. To validation I've put 28 samples.

The problem is that my validation loss and accuracy is unstable. This is my code:

def ReadImages(Path):
    LabelList = list()
    ImageCV = list()
    classes = ["nonPdr", "pdr"]

    # Get all subdirectories
    FolderList = [f for f in os.listdir(Path) if not f.startswith('.')]

    # Loop over each directory
    for File in FolderList:
        for index, Image in enumerate(os.listdir(os.path.join(Path, File))):
            # Convert the path into a file
            ImageCV.append(cv2.resize(cv2.imread(os.path.join(Path, File) + os.path.sep + Image), (256,256)))
            #ImageCV[index]= np.array(ImageCV[index]) / 255.0
            LabelList.append(classes.index(os.path.splitext(File)[0])) 


            ImageCV[index] = cv2.addWeighted(ImageCV[index],4, cv2.GaussianBlur(ImageCV[index],(0,0), 256/30), -4, 128)


    return ImageCV, LabelList

visible = Input(shape=(256,256,3))
conv1 = Conv2D(16, kernel_size=(3,3), activation='relu', strides=(1, 1))(visible)
conv2 = Conv2D(16, kernel_size=(3,3), activation='relu', strides=(1, 1))(conv1)
bat1 = BatchNormalization()(conv2)
conv3 = ZeroPadding2D(padding=(1, 1))(bat1)
pool1 = MaxPooling2D(pool_size=(2, 2))(conv3)

conv4 = Conv2D(32, kernel_size=(3,3), activation='relu', padding='valid', kernel_regularizer=regularizers.l2(0.01))(pool1)
conv5 = Conv2D(32, kernel_size=(3,3), activation='relu', padding='valid', kernel_regularizer=regularizers.l2(0.01))(conv4)
bat2 = BatchNormalization()(conv5)
pool2 = MaxPooling2D(pool_size=(1, 1))(bat2)

conv6 = Conv2D(64, kernel_size=(3,3), activation='relu',strides=(1, 1), padding='valid')(pool2)
conv7 = Conv2D(64, kernel_size=(3,3), activation='relu',strides=(1, 1), padding='valid')(conv6)
bat3 = BatchNormalization()(conv7)
conv7 = ZeroPadding2D(padding=(1, 1))(bat3)
pool3 = MaxPooling2D(pool_size=(1, 1))(conv7)

conv8 = Conv2D(128, kernel_size=(3,3), activation='relu', padding='valid', kernel_regularizer=regularizers.l2(0.01))(pool3)
conv9 = Conv2D(128, kernel_size=(2,2), activation='relu', strides=(1, 1), padding='valid')(conv8)
bat4 = BatchNormalization()(conv9)
pool4 = MaxPooling2D(pool_size=(1, 1))(bat4)

conv10 = Conv2D(256, kernel_size=(3,3), activation='relu', padding='valid', kernel_regularizer=regularizers.l2(0.02))(pool4)
conv11 = Conv2D(256, kernel_size=(3,3), activation='relu', padding='valid', kernel_regularizer=regularizers.l2(0.02))(conv10)
bat5 = BatchNormalization()(conv11)
pool5 = MaxPooling2D(pool_size=(1, 1))(bat5)

flat = Flatten()(pool5)

output = Dense(1, activation='sigmoid')(flat)
model = Model(inputs=visible, outputs=output)

opt = optimizers.adam(lr=0.001, decay=0.0)

model.compile(optimizer= opt, loss='binary_crossentropy', metrics=['accuracy'])



data, labels = ReadImages(TRAIN_DIR)

test, lt = ReadImages(TEST_DIR)

model.fit(np.array(data), np.array(labels), epochs=8, validation_data = (np.array(test), np.array(lt)))

model.save('model.h5')

And after run it, I've got the follow return:

Train on 1400 samples, validate on 28 samples
Epoch 1/8
1400/1400 [==============================] - 2289s 2s/step - loss: 10.5126 - acc: 0.9529 - val_loss: 9.6115 - val_acc: 1.0000
Epoch 2/8
1400/1400 [==============================] - 2245s 2s/step - loss: 9.8477 - acc: 0.9550 - val_loss: 9.1845 - val_acc: 0.9643
Epoch 3/8
1400/1400 [==============================] - 2271s 2s/step - loss: 8.3761 - acc: 0.9864 - val_loss: 7.6834 - val_acc: 1.0000
Epoch 4/8
1400/1400 [==============================] - 2225s 2s/step - loss: 7.7146 - acc: 0.9736 - val_loss: 15.1970 - val_acc: 0.5000
Epoch 5/8
1400/1400 [==============================] - 2204s 2s/step - loss: 7.8170 - acc: 0.9436 - val_loss: 6.5526 - val_acc: 1.0000
Epoch 6/8
1400/1400 [==============================] - 2215s 2s/step - loss: 7.1557 - acc: 0.9407 - val_loss: 5.8400 - val_acc: 1.0000
Epoch 7/8
1400/1400 [==============================] - 2267s 2s/step - loss: 6.4109 - acc: 0.9450 - val_loss: 5.2029 - val_acc: 1.0000
Epoch 8/8
1400/1400 [==============================] - 2269s 2s/step - loss: 5.8860 - acc: 0.9479 - val_loss: 13.2224 - val_acc: 0.5000

And when I try to predict some test's images, All of the samples returns the class 0 (wrong): PREDICT.py

model = load_model('model.h5')

for filename in os.listdir(r'v/'):
    if filename.endswith(".jpg") or filename.endswith(".png"):
        ImageCV = cv2.resize(cv2.imread(os.path.join(TEST_DIR) + os.path.sep + filename), (256,256))
        ImageCV = cv2.addWeighted(ImageCV,4, cv2.GaussianBlur(ImageCV,(0,0), 256/30), -4, 128)
        ImageCV = ImageCV.reshape(-1,256,256,3)
        print(model.predict(ImageCV))
        print(np.argmax(model.predict(ImageCV)))
[[0.]]
0
[[0.]]
0
[[0.]]
0

So, what I'm doing wrong in my project? How can I fix it?

I appreciate any help

UPDATE

After add this code:

perm = np.random.permutation(len(data))
data = data[perm]
labels = labels[perm]

Returns this numbers:

Train on 1400 samples, validate on 28 samples
Epoch 1/5
1400/1400 [==============================] - 2232s 2s/step - loss: 10.5725 - acc: 0.9629 - val_loss: 10.1279 - val_acc: 1.0000
Epoch 2/5
1400/1400 [==============================] - 2370s 2s/step - loss: 10.2828 - acc: 0.9729 - val_loss: 9.5293 - val_acc: 1.0000
Epoch 3/5
1400/1400 [==============================] - 2290s 2s/step - loss: 9.6735 - acc: 0.9707 - val_loss: 8.8646 - val_acc: 1.0000
Epoch 4/5
1400/1400 [==============================] - 2269s 2s/step - loss: 8.6198 - acc: 0.9950 - val_loss: 8.1976 - val_acc: 1.0000
Epoch 5/5
1400/1400 [==============================] - 2282s 2s/step - loss: 8.2455 - acc: 0.9836 - val_loss: 7.8586 - val_acc: 1.0000

The value goes better, but when I try to predict images, the return is always 0.. (but i'm passing imgs class 0 and class 1)

What should I do now?

$\endgroup$
1
$\begingroup$

I guess your model is biased toward the first half. Although Keras has built in shuffle=True in model.fit() arguments, according to this document it might be non-effective when steps_per_epoch=None.

I suggest shuffling your data before training using numpy.random.shuffle(array). Probably something like this:

data = np.array(data)
labels = np.array(labels)

perm = np.random.permutation(len(data))
data = data[perm]
labels = labels[perm]
model.fit(data, labels, epochs=8, validation_data = (np.array(test), np.array(lt)))

| improve this answer | |
$\endgroup$
  • $\begingroup$ Should I shuffle which array? $\endgroup$ – Gilberto Sep 14 '19 at 1:20
  • 1
    $\begingroup$ Image and Label arrays should be shuffled together before calling .fit() function. Both of them should be shuffled with a single pattern so that correct labels assigned to each image $\endgroup$ – aminrd Sep 14 '19 at 1:24
  • $\begingroup$ np.random.shuffle(data) np.random.shuffle(labels) np.random.shuffle(test) np.random.shuffle(lt) $\endgroup$ – Gilberto Sep 15 '19 at 17:18
  • $\begingroup$ I should be like this? $\endgroup$ – Gilberto Sep 15 '19 at 17:19
  • $\begingroup$ In order to shuffle two arrays a and b in ordered place, you can use: p = numpy.random.permutation(len(a)) to generate permutation and then shuffle them using a = a[p] and b = b[p] $\endgroup$ – aminrd Sep 15 '19 at 21:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.