0
$\begingroup$

I have a binary classification problem. I get ROI mammogram images and then apply a decomposition algorithm and as output I get 5 images which summation of them results in the original image. Now, having these five components, I apply a VGG-like deep model for training a network, considering an input shape as (320,320,5) in which I feed five components as different input channels. Below is the code for my network:

def MyNet():
    from keras.optimizers import Adam
    regularization_rate = 0.01
    model = Sequential()
    model.add(Conv2D(input_shape=(320, 320, 5), filters=64, kernel_size=(3, 3), padding="same", activation="relu"))
    model.add(BatchNormalization())
    model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))
    model.add(BatchNormalization())
    model.add(MaxPooling2D((2, 2), strides=(2, 2)))

    # Block 2
    model.add(Conv2D(128, (3, 3), activation='relu', padding='same'))
    model.add(BatchNormalization())
    model.add(Conv2D(128, (3, 3), activation='relu', padding='same'))
    model.add(BatchNormalization())
    model.add(MaxPooling2D((2, 2), strides=(2, 2)))

    # Block 3
    model.add(Conv2D(256, (3, 3), activation='relu', padding='same'))
    model.add(BatchNormalization())
    model.add(Conv2D(256, (3, 3), activation='relu', padding='same'))
    model.add(BatchNormalization())
    model.add(Conv2D(256, (3, 3), activation='relu', padding='same'))
    model.add(BatchNormalization())
    model.add(MaxPooling2D((2, 2), strides=(2, 2)))

    # Block 4
    model.add(Conv2D(512, (3, 3), activation='relu', padding='same'))
    model.add(BatchNormalization())
    model.add(Conv2D(512, (3, 3), activation='relu', padding='same'))
    model.add(BatchNormalization())
    model.add(Conv2D(512, (3, 3), activation='relu', padding='same'))
    model.add(BatchNormalization())
    model.add(MaxPooling2D((2, 2), strides=(2, 2)))

    # Block 5
    model.add(Conv2D(512, (3, 3), activation='relu', padding='same'))
    model.add(BatchNormalization())
    model.add(Conv2D(512, (3, 3), activation='relu', padding='same'))
    model.add(BatchNormalization())
    model.add(Conv2D(512, (3, 3), activation='relu', padding='same'))
    model.add(BatchNormalization())
    model.add(MaxPooling2D((2, 2), strides=(2, 2)))

    # Flatten and fully connected layers
    model.add(Flatten())
    model.add(Dense(4096, activation='relu', kernel_regularizer=l2(regularization_rate)))
    model.add(BatchNormalization())
    model.add(Dense(4096, activation='relu', kernel_regularizer=l2(regularization_rate)))
    model.add(BatchNormalization())

    # Output layer for binary classification
    model.add(Dense(1, activation='sigmoid'))

    # Compile the model
    model.compile(optimizer='adam', loss='binary_crossentropy',
                  metrics=['accuracy', 'AUC', f1_m, recall_m, precision_m])
    return (model)

And below is the code for training and validation using 5-fold cross validation:

def kfold_indices(data, k):
    fold_size = len(data) // k
    indices = np.arange(len(data))
    folds = []
    for i in range(k):
        val_indices = indices[i * fold_size: (i+1) * fold_size]
        train_indices = np.concatenate([indices[:i * fold_size], indices[(i+1) * fold_size:]])
        folds.append((train_indices, val_indices))
    return folds


k = 5
fold_indices = kfold_indices(X_train, k)
accuracies = []
index_fold = 1

for train_indices, val_indices in fold_indices:
    print("===============================", index_fold, "===========================================")
    X_train_fold, y_train_fold = X_train[train_indices], y_train[train_indices]

    X_val, y_val = X_train[val_indices], y_train[val_indices]

    # fittiing the model  on X_train and y_train --> K-fold
    model = MyNet()
    # model.compile(loss='binary_crossentropy', optimizer='RMSprop', metrics=['accuracy', 'AUC'])
    train_res = model.fit(X_train_fold, y_train_fold, epochs=50, batch_size=32, validation_data=(X_val, y_val),
                          validation_batch_size=32, verbose=1, callbacks=[temp_logger])
    accuracies.append(train_res)
    index_fold = index_fold + 1

The input X_train is in shape (1303, 320, 320, 5). The output is all folds are quite similar as below for the last 10 epochs:

Epoch 40/50
33/33 [==============================] - 10s 297ms/step - loss: 1.5994 - accuracy: 0.9310 - auc: 0.9771 - f1_m: 0.9352 - recall_m: 0.9474 - precision_m: 0.9288 - val_loss: 2.3136 - val_accuracy: 0.5385 - val_auc: 0.5847 - val_f1_m: 0.4224 - val_recall_m: 0.3604 - val_precision_m: 0.5441
Epoch 41/50
33/33 [==============================] - 10s 296ms/step - loss: 1.7654 - accuracy: 0.9060 - auc: 0.9602 - f1_m: 0.9013 - recall_m: 0.9089 - precision_m: 0.9058 - val_loss: 3.3083 - val_accuracy: 0.5231 - val_auc: 0.5456 - val_f1_m: 0.5832 - val_recall_m: 0.7177 - val_precision_m: 0.5242
Epoch 42/50
33/33 [==============================] - 10s 296ms/step - loss: 1.9983 - accuracy: 0.8974 - auc: 0.9504 - f1_m: 0.9001 - recall_m: 0.9098 - precision_m: 0.8982 - val_loss: 3.3215 - val_accuracy: 0.5115 - val_auc: 0.6062 - val_f1_m: 0.3833 - val_recall_m: 0.3506 - val_precision_m: 0.6310
Epoch 43/50
33/33 [==============================] - 10s 297ms/step - loss: 1.5076 - accuracy: 0.9291 - auc: 0.9785 - f1_m: 0.9303 - recall_m: 0.9401 - precision_m: 0.9278 - val_loss: 3.3200 - val_accuracy: 0.5346 - val_auc: 0.6354 - val_f1_m: 0.4369 - val_recall_m: 0.3833 - val_precision_m: 0.6552
Epoch 44/50
33/33 [==============================] - 10s 300ms/step - loss: 1.3974 - accuracy: 0.9348 - auc: 0.9816 - f1_m: 0.9363 - recall_m: 0.9352 - precision_m: 0.9443 - val_loss: 2.7090 - val_accuracy: 0.5846 - val_auc: 0.6078 - val_f1_m: 0.6341 - val_recall_m: 0.7522 - val_precision_m: 0.5820
Epoch 45/50
33/33 [==============================] - 10s 298ms/step - loss: 1.3208 - accuracy: 0.9185 - auc: 0.9742 - f1_m: 0.9194 - recall_m: 0.9292 - precision_m: 0.9169 - val_loss: 1.8723 - val_accuracy: 0.6538 - val_auc: 0.6667 - val_f1_m: 0.6646 - val_recall_m: 0.7490 - val_precision_m: 0.6422
Epoch 46/50
33/33 [==============================] - 10s 296ms/step - loss: 1.1986 - accuracy: 0.9482 - auc: 0.9866 - f1_m: 0.9498 - recall_m: 0.9463 - precision_m: 0.9584 - val_loss: 2.1230 - val_accuracy: 0.6346 - val_auc: 0.7082 - val_f1_m: 0.6837 - val_recall_m: 0.8356 - val_precision_m: 0.6041
Epoch 47/50
33/33 [==============================] - 10s 297ms/step - loss: 1.4068 - accuracy: 0.9367 - auc: 0.9869 - f1_m: 0.9380 - recall_m: 0.9429 - precision_m: 0.9421 - val_loss: 4.4375 - val_accuracy: 0.5885 - val_auc: 0.6231 - val_f1_m: 0.5886 - val_recall_m: 0.6213 - val_precision_m: 0.6146
Epoch 48/50
33/33 [==============================] - 10s 297ms/step - loss: 1.6808 - accuracy: 0.9070 - auc: 0.9637 - f1_m: 0.9073 - recall_m: 0.9208 - precision_m: 0.9032 - val_loss: 4.6520 - val_accuracy: 0.4846 - val_auc: 0.5533 - val_f1_m: 0.1789 - val_recall_m: 0.1086 - val_precision_m: 0.5926
Epoch 49/50
33/33 [==============================] - 10s 297ms/step - loss: 1.4291 - accuracy: 0.9291 - auc: 0.9765 - f1_m: 0.9305 - recall_m: 0.9386 - precision_m: 0.9301 - val_loss: 2.6339 - val_accuracy: 0.5885 - val_auc: 0.6236 - val_f1_m: 0.6426 - val_recall_m: 0.7774 - val_precision_m: 0.5738
Epoch 50/50
33/33 [==============================] - 10s 297ms/step - loss: 1.7352 - accuracy: 0.9003 - auc: 0.9643 - f1_m: 0.8992 - recall_m: 0.8958 - precision_m: 0.9141 - val_loss: 2.2339 - val_accuracy: 0.5885 - val_auc: 0.6073 - val_f1_m: 0.6624 - val_recall_m: 0.8475 - val_precision_m: 0.5640

As it can be seen there is a huge difference between training and validation results. I know there can be overfitting problem. As I searched through literature I found and tested the below options:

  1. adding dropout layer:modifying the last layers as
    model.add(Flatten())
    model.add(Dense(4096, activation='relu', kernel_regularizer=l2(regularization_rate)))
    model.add(BatchNormalization())
    model.add(Dropout(0.5))  # Add dropout with rate 0.5
    model.add(Dense(4096, activation='relu', kernel_regularizer=l2(regularization_rate)))
    model.add(BatchNormalization())
    model.add(Dropout(0.5))  # Add dropout with rate 0.5
  1. adding regularization:
model.add(Flatten())
    model.add(Dense(4096, activation='relu', kernel_regularizer=l2(regularization_rate)))
    model.add(BatchNormalization())
    model.add(Dense(4096, activation='relu', kernel_regularizer=l2(regularization_rate)))
    model.add(BatchNormalization())
  1. changing batch size: changing training batch size from 32 to 16 and 64
  2. changing loss function: changing from binary cross entropy to
tensorflow.keras.losses.BinaryFocalCrossentropy(gamma=2.0, from_logits=True)
  1. using different activation functions: tried with tanh

Doing all these, the results did not change for validation and in some cases it destroyed the training results as well.

What is the main reason behind this? Is the network design appropriate? and how should I address my problem?

I would really appreciate any help.

*A note that as suggested below, I tried with ResNet50. However, the results for training only changed not the validation. I have below the last five epochs results:

Epoch 45/50
33/33 [==============================] - 6s 183ms/step - loss: 2.8129e-05 - accuracy: 1.0000 - auc: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 2.0236 - val_accuracy: 0.6077 - val_auc: 0.6410 - val_f1_m: 0.5755 - val_precision_m: 0.5821 - val_recall_m: 0.5809
Epoch 46/50
33/33 [==============================] - 6s 184ms/step - loss: 3.0869e-05 - accuracy: 1.0000 - auc: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 2.0310 - val_accuracy: 0.6077 - val_auc: 0.6413 - val_f1_m: 0.5755 - val_precision_m: 0.5821 - val_recall_m: 0.5809
Epoch 47/50
33/33 [==============================] - 6s 183ms/step - loss: 2.9120e-05 - accuracy: 1.0000 - auc: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 2.0403 - val_accuracy: 0.6077 - val_auc: 0.6413 - val_f1_m: 0.5755 - val_precision_m: 0.5821 - val_recall_m: 0.5809
Epoch 48/50
33/33 [==============================] - 6s 184ms/step - loss: 2.8139e-05 - accuracy: 1.0000 - auc: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 2.0449 - val_accuracy: 0.6038 - val_auc: 0.6425 - val_f1_m: 0.5737 - val_precision_m: 0.5797 - val_recall_m: 0.5809
Epoch 49/50
33/33 [==============================] - 6s 184ms/step - loss: 2.9613e-05 - accuracy: 1.0000 - auc: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 2.0428 - val_accuracy: 0.6077 - val_auc: 0.6423 - val_f1_m: 0.5755 - val_precision_m: 0.5821 - val_recall_m: 0.5809
Epoch 50/50
33/33 [==============================] - 6s 183ms/step - loss: 3.0790e-05 - accuracy: 1.0000 - auc: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 2.0455 - val_accuracy: 0.6077 - val_auc: 0.6395 - val_f1_m: 0.5755 - val_precision_m: 0.5821 - val_recall_m: 0.5809

I am really stuck with this and any help would be really appreciated.

$\endgroup$

1 Answer 1

1
$\begingroup$

You are using medical images for which the difference in images might not be significant. For this, you can use residual networks or ResNet architecture which has residual connections which improves generalizability.

You can use ResNet50 for a start and progress to large layered ResNets such as ResNet100 or ResNet150.

$\endgroup$
1
  • $\begingroup$ Hi @Rathod. Thanks for your suggestion. I have updated the original post for ResNet50 results. As you can see ResNet also did not work. $\endgroup$
    – Nmgh
    Dec 14, 2023 at 20:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.