0
$\begingroup$

I have a binary classification problem. I designed a model with convolution kernels in first layers and then dense layers. As the output layer, however, I used a softmax layer with size 2, and then used one-hot encoding on my labels. It means my labels look like a 2-bit number: either 01 or 10.

Also, I report the following metrics: fp, fn, tp, tn, recall, precision and accuracy.

My problem is: I get really strange results during training as seen below. Accuracy, precision and recall are always equal. To be more specific: fp=fn, tp=tn, recall=precision=accuracy!!

Can somebody explain that? and how to fix it?

Here is my code to preprocess the data, build the model and fit.

def preprocess_data(X_train, y_train, X_test, y_test):
    X_train = X_train.reshape(X_train.shape[0], X_train.shape[1], X_train.shape[2], 1)
    X_test = X_test.reshape(X_test.shape[0], X_test.shape[1], X_train.shape[2], 1)

    X_train = X_train.astype('float32')
    X_test = X_test.astype('float32')

    # normalization
    max_signal_value = max(X_train.max(), X_test.max())
    X_train = X_train/max_signal_value 
    X_test = X_test/max_signal_value 

    # make binary
    y_train = np.array(list(map(lambda x:int(x==1), y_train)))
    y_test = np.array(list(map(lambda x:int(x==1), y_test)))

    # One-hot encoding label
    y_train = to_categorical(y_train)
    y_test = to_categorical(y_test)

    return X_train, y_train, X_test, y_test

metrics = [
    keras.metrics.FalseNegatives(name="fn"),
    keras.metrics.FalsePositives(name="fp"),
    keras.metrics.TrueNegatives(name="tn"),
    keras.metrics.TruePositives(name="tp"),
    keras.metrics.Precision(name="precision"),
    keras.metrics.Recall(name="recall"),
    'accuracy'
]

def build_model():
    model = Sequential()

    model.add(Conv2D(filters = 6, kernel_size = (5,5), padding = 'same',
                   activation = 'relu', input_shape = (28,28,1)))


    model.add(MaxPooling2D(pool_size=(2, 2)))

    model.add(Conv2D(filters=16, kernel_size=(5, 5), activation='relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Flatten())

    model.add(Dense(20, activation='relu'))
    model.add(Dense(2, activation='softmax'))

    opt = Adam(learning_rate=0.001)
    model.compile(loss=categorical_crossentropy, optimizer=opt, metrics=metrics)

    return model

LeNet_model = build_model()
LeNet_model.summary()

X_train, y_train, X_test, y_test = preprocess_data(X_train, y_train, X_test, y_test)

history = LeNet_model.fit(X_train, y_train, epochs=50, batch_size=1,
                        steps_per_epoch=X_train.shape[0], validation_data=(X_test, y_test),
                        validation_steps=X_test.shape[0], verbose=1)

The result looks like:

74008/74008 [=======] - loss: 0.5173 - fn: 16423.0000 - fp: 16423.0000 - tn: 57585.0000 - tp: 57585.0000 - precision: 0.7781 - recall: 0.7781 - accuracy: 0.7781 - cat_accuracy: 0.7781 - val_loss: 0.5173 - val_fn: 4643.0000 - val_fp: 4643.0000 - val_tn: 15487.0000 - val_tp: 15487.0000 - val_precision: 0.7693 - val_recall: 0.7693 - val_accuracy: 0.7693 - val_cat_accuracy: 0.7693
Epoch 2/50

74008/74008 [=======] - loss: 0.4986 - fn: 16404.0000 - fp: 16404.0000 - tn: 57604.0000 - tp: 57604.0000 - precision: 0.7783 - recall: 0.7783 - accuracy: 0.7783 - cat_accuracy: 0.7783 - val_loss: 0.5200 - val_fn: 4644.0000 - val_fp: 4644.0000 - val_tn: 15486.0000 - val_tp: 15486.0000 - val_precision: 0.7693 - val_recall: 0.7693 - val_accuracy: 0.7693 - val_cat_accuracy: 0.7693```
$\endgroup$

1 Answer 1

0
$\begingroup$

It's a binary classification, you should change the last layers to 1 unit with sigmoid activation. The loss function also should change to binary_crossentropy instead.

Also, for the y, just keep its value as 0 or 1, no need to do the One-Hot Encoding to make it become 01 or 10. It's redundant and not recommended.

You can try this code for the model building:

def build_model():
    model = Sequential()

    model.add(Conv2D(filters = 6, kernel_size = (5,5), padding = 'same',
                   activation = 'relu', input_shape = (28,28,1)))


    model.add(MaxPooling2D(pool_size=(2, 2)))

    model.add(Conv2D(filters=16, kernel_size=(5, 5), activation='relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Flatten())

    model.add(Dense(20, activation='relu'))
    model.add(Dense(1, activation='sigmoid'))

    opt = Adam(learning_rate=0.001)
    model.compile(loss='binary_crossentropy', optimizer=opt, metrics=metrics)

return model

Hope this help.

$\endgroup$
1
  • $\begingroup$ Actually, I've already done that. The results were not good. Therefore, I thought maybe categorical approach would work better. The question here is, why doesn't this code work? It seems logical to me: softmax with 2 output should work just like sigmoid with 1 output. Or do I miss something? $\endgroup$
    – Farzad
    Nov 17, 2022 at 10:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.