0
$\begingroup$

As the title clearly describes the issue I've been experiencing during the training of my CNN model, the accuracies of training and validation sets are constant despite the losses of them are changing. I have included the detail regarding the model and its training setup below. What may cause this issue?

Here is the data that was used by training (X_train & y_train), validation, and test sets (X_test and y_test):

df = pd.read_csv(CSV_PATH, sep=',', header=None)
print(f'Shape of all data: {df.shape}')
    
y = df.iloc[:, -1].values
X = df.iloc[:, :-1].values

encoder = LabelEncoder()
encoder.fit(y)
encoded_Y = encoder.transform(y)
dummy_y = to_categorical(encoded_Y)

X_train, X_test, y_train, y_test = train_test_split(X, dummy_y, test_size=0.3, random_state=RANDOM_STATE)

X_train = X_train.reshape((X_train.shape[0], X_train.shape[1], 1))
X_test = X_test.reshape((X_test.shape[0], X_test.shape[1], 1))

Here are the shapes of training and test sets:

Shape of X_train: (1322, 10800, 1)
Shape of Y_train: (1322, 3)
Shape of X_test: (567, 10800, 1)
Shape of y_test: (567, 3)

Here is my CNN model:

# Model hyper-parameters
activation_fn = 'relu'
n_lr = 1e-4
weight_decay = 1e-4
batch_size = 64
num_epochs = 200*10*10
num_classes = 3
n_dropout = 0.6
n_momentum = 0.5
n_kernel = 5
n_reg = 1e-5

# the sequential model
model = Sequential()
model.add(Conv1D(128, n_kernel, input_shape=(10800, 1)))
model.add(BatchNormalization())
model.add(Activation(activation_fn))
model.add(MaxPooling1D(pool_size=2, strides=2))

model.add(Dropout(n_dropout))

model.add(Conv1D(256, n_kernel))
model.add(BatchNormalization())
model.add(Activation(activation_fn))
model.add(MaxPooling1D(pool_size=2, strides=2))

model.add(Dropout(n_dropout))

model.add(GlobalAveragePooling1D()) # have tried model.add(Flatten()) as well

model.add(Dense(256, activation=activation_fn))
model.add(Dropout(n_dropout))
model.add(Dense(64, activation=activation_fn))
model.add(Dropout(n_dropout))
model.add(Dense(num_classes, activation='softmax'))

adam = Adam(lr=n_lr, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=weight_decay)
model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['acc'])

Here is how I have evaluated the model:

Y_pred = model.predict(X_test, verbose=0)
y_pred = np.argmax(Y_pred, axis=1)
y_test_int = np.argmax(y_test, axis=1)

And, my model always predicts the same class of three classes during the model evaluation as you can see from the classification result below (via classification_result(y_test_int, y_pred) function):

          precision    recall  f1-score   support

  normal      0.743     1.000     0.852       421
     apb      0.000     0.000     0.000        45
     pvc      0.000     0.000     0.000       101

The model was trained using the EarlyStopping callback of Keras. So, the training has continued for 4,173 epochs. Here is the obtained losses during the training for training and validation sets:

loss

Here is the obtained accuracies during the training for training and validation sets:

accuracy

The model was implemented using Keras, and hosted on Google Colab. Please feel free to ask any further information regarding my model.

$\endgroup$
4
  • 2
    $\begingroup$ There could be a number of reasons why your model isn't improving. In the 1st Conv1D layer, the input shape is ( 10800 , 1 ). If you're trying to classify such a long sequence, I'd insist you add more Conv1D layers. Probably your model doesn't have that many parameters to approximate the input vs. output relation. $\endgroup$ – Shubham Panchal Apr 6 at 2:00
  • $\begingroup$ Thanks for your comment, @ShubhamPanchal. To be honest, I've already increased the number of Conv1D layers but the result was the same as I oddly got constant accuracy values for training and validation sets despite the changes in the loss values. $\endgroup$ – talha06 Apr 6 at 10:58
  • $\begingroup$ You can try decaying the learning rate after a specific number of steps/epochs. So, as the loss function approaches the global minima, the learning rate is reduced according to a given scheme ( as discussed here ) and it will avoid skipping the minima. $\endgroup$ – Shubham Panchal Apr 8 at 7:13
  • $\begingroup$ I've employed the ReduceLROnPlateau callback as follows but nothing is changed: ReduceLROnPlateau(monitor='val_loss', factor=0.1, patience=2, min_lr=0.000001, verbose=1). Also, tried LearningRateScheduler as well. It did not change the issue, too. Is there anything you can recommend me to try? $\endgroup$ – talha06 Apr 8 at 13:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.