# CNN model low accuracy

I have 1299 images in 4 classes (374/269/284/372). I want to use the VGG19 model, add a dense layer at the top and fine-tune it with my images. As I only have 1299 images, I also want to use data augmentation.

Here is the code (it is not exactly the code I used everytime, as I tried several things. When I did not use preprocess_input, I did X = X/255.) :

drive.mount('/content/gdrive')

CATEGORIES = ['A','B','C','D']
training_data = []

def create_training_data():
for category in CATEGORIES:
class_num = CATEGORIES.index(category)
for img in tqdm(os.listdir(path)):
img_array_RGB = cv2.cvtColor(img_array, cv2.COLOR_BGR2RGB)
new_array = cv2.resize(img_array_RGB, (224,224))
training_data.append([new_array,class_num])

create_training_data()

X = []
Y = []

for features, label, in training_data:
X.append(features)
Y.append(label)

X = np.array(X)
Y = np.array(Y)

IMG_SIZE=224
X = np.array(X).reshape(-1, IMG_SIZE, IMG_SIZE, 3)

Y_class = Y
Y = to_categorical(Y, num_classes=4)

X_train, X_val, Y_train, Y_val = train_test_split(X, Y, test_size = 0.3, random_state=0, stratify=Y)

Y_train_int, Y_val_int = [np.where(r==1)[0][0] for r in Y_train], [np.where(r==1)[0][0] for r in Y_val]

from keras.preprocessing.image import ImageDataGenerator
datagen = ImageDataGenerator(
horizontal_flip=True,
vertical_flip=True,
brightness_range=[0.2,0.6],
fill_mode='wrap')

datagen.fit(X_train)

vgg_model = VGG19(weights='imagenet', include_top=False)

X = preprocess_input(X)

x = vgg_model.output
x = GlobalAveragePooling2D()(x)

x = Dense(512, activation='relu')(x)
x = Dropout(0.3)(x)

predictions = Dense(4, activation='softmax')(x)

model = Model(inputs=vgg_model.input, outputs=predictions)

for layer in vgg_model.layers:
layer.trainable = False

layer_num = len(model.layers)
for layer in model.layers[:21]:
layer.trainable = False

for layer in model.layers[21:]:
layer.trainable = True

#class_weights = class_weight.compute_class_weight('balanced',
#                                                 np.unique(Y_train_int),
#                                                 Y_train_int)

history = model.fit(datagen.flow(X_train,Y_train, batch_size=64), epochs=30, batch_size=128, shuffle=True, validation_data = (X_val,Y_val))



However this does not produce relevant results. I tried several modifications (changing the learning rate, batch size, loss as sparse_categorical_crossentropy by changing from one hot encoding to integer, adding weights to classes, and I also tried to build a CNN from scratch with keras, but nothing seems to give better results (predicting the class with the most images)

Here is an example of plots : https://i.stack.imgur.com/KODAr.png

I don't really know what else I could try, but I really think a good model could classify these images better than this.

Edit : I changed some parameters :

• Data augmentation (I deleted the brightness)
• batch_size = 32 for the data augmentation and 64 for the model
• added weights to the classes
• added x = Dense(256, activation='relu')(x) after x = Dense(512, activation='relu')(x)

And now the learning curve and confusion matrix look like this (only 10 epochs) : https://i.stack.imgur.com/IsJTh.png

The learning curve is really fluctuating and the model doesn't seem to learn, but the accuracy is higher than before. I don't know what else to do now.