I have 1299 images in 4 classes (374/269/284/372). I want to use the VGG19 model, add a dense layer at the top and fine-tune it with my images. As I only have 1299 images, I also want to use data augmentation.
Here is the code (it is not exactly the code I used everytime, as I tried several things. When I did not use preprocess_input, I did X = X/255.) :
drive.mount('/content/gdrive') DATADIR = '/content/gdrive/My Drive/' CATEGORIES = ['A','B','C','D'] training_data =  def create_training_data(): for category in CATEGORIES: path = os.path.join(DATADIR,category) class_num = CATEGORIES.index(category) for img in tqdm(os.listdir(path)): img_array = cv2.imread(os.path.join(path,img)) img_array_RGB = cv2.cvtColor(img_array, cv2.COLOR_BGR2RGB) new_array = cv2.resize(img_array_RGB, (224,224)) training_data.append([new_array,class_num]) create_training_data() X =  Y =  for features, label, in training_data: X.append(features) Y.append(label) X = np.array(X) Y = np.array(Y) IMG_SIZE=224 X = np.array(X).reshape(-1, IMG_SIZE, IMG_SIZE, 3) Y_class = Y Y = to_categorical(Y, num_classes=4) X_train, X_val, Y_train, Y_val = train_test_split(X, Y, test_size = 0.3, random_state=0, stratify=Y) Y_train_int, Y_val_int = [np.where(r==1) for r in Y_train], [np.where(r==1) for r in Y_val] from keras.preprocessing.image import ImageDataGenerator datagen = ImageDataGenerator( horizontal_flip=True, vertical_flip=True, brightness_range=[0.2,0.6], fill_mode='wrap') datagen.fit(X_train) vgg_model = VGG19(weights='imagenet', include_top=False) X = preprocess_input(X) x = vgg_model.output x = GlobalAveragePooling2D()(x) x = Dense(512, activation='relu')(x) x = Dropout(0.3)(x) predictions = Dense(4, activation='softmax')(x) model = Model(inputs=vgg_model.input, outputs=predictions) for layer in vgg_model.layers: layer.trainable = False layer_num = len(model.layers) for layer in model.layers[:21]: layer.trainable = False for layer in model.layers[21:]: layer.trainable = True #class_weights = class_weight.compute_class_weight('balanced', # np.unique(Y_train_int), # Y_train_int) model.compile(optimizer='Adam', loss='categorical_crossentropy', metrics=['accuracy']) history = model.fit(datagen.flow(X_train,Y_train, batch_size=64), epochs=30, batch_size=128, shuffle=True, validation_data = (X_val,Y_val))
However this does not produce relevant results. I tried several modifications (changing the learning rate, batch size, loss as sparse_categorical_crossentropy by changing from one hot encoding to integer, adding weights to classes, and I also tried to build a CNN from scratch with keras, but nothing seems to give better results (predicting the class with the most images)
Here is an example of plots : https://i.stack.imgur.com/KODAr.png
I don't really know what else I could try, but I really think a good model could classify these images better than this.
Edit : I changed some parameters :
- Data augmentation (I deleted the brightness)
- batch_size = 32 for the data augmentation and 64 for the model
- added weights to the classes
- added x = Dense(256, activation='relu')(x) after x = Dense(512, activation='relu')(x)
And now the learning curve and confusion matrix look like this (only 10 epochs) : https://i.stack.imgur.com/IsJTh.png
The learning curve is really fluctuating and the model doesn't seem to learn, but the accuracy is higher than before. I don't know what else to do now.