0
$\begingroup$

I run a binary classification using different CNN versions in Tensorflow. When I label samples from each class using 0 and 1, I select a sigmoid output in the last layer of the CNN, like

 x = Dense(1, activation="sigmoid")(x)

Then, I set up a series of metrics to follow training progress, as in

metrics = [
  keras.metrics.TruePositives(name='tp'),
  keras.metrics.FalsePositives(name='fp'),
  keras.metrics.TrueNegatives(name='tn'),
  keras.metrics.FalseNegatives(name='fn'), 
  keras.metrics.BinaryAccuracy(name='accuracy'),
  keras.metrics.Precision(name='precision'),
  keras.metrics.Recall(name='recall'),
  keras.metrics.AUC(name='auc'),
  keras.metrics.AUC(name='prc', curve='PR'), # precision-recall curve]

Finally, my model is compiled and fit using binary crossentropy loss:

# Compile the model
model.compile(optimizer=Adam(), loss='binary_crossentropy', metrics=metrics)
history = model.fit(X_train, y_train, 
                         batch_size=batch_size, epochs=number_of_epochs, 
                         verbose=verbosity_mode,
                         validation_data = (X_val, y_val),
                         callbacks=[es, cc])

Everything works fine, as I can see the metrics both for my training and my validation set changing during training:

Epoch 58/60
132/132 - 24s - loss: 0.0205 - tp: 2413.0000 - fp: 9.0000 - tn: 1770.0000 - fn: 7.0000 - accuracy: 0.9962 - precision: 0.9963 - recall: 0.9971 - auc: 0.9999 - prc: 0.9999 - val_loss: 0.3301 - val_tp: 1402.0000 - val_fp: 182.0000 - val_tn: 2039.0000 - val_fn: 178.0000 - val_accuracy: 0.9053 - val_precision: 0.8851 - val_recall: 0.8873 - val_auc: 0.9607 - val_prc: 0.9426 - 24s/epoch - 180ms/step
Epoch 59/60
132/132 - 25s - loss: 0.0217 - tp: 2408.0000 - fp: 11.0000 - tn: 1768.0000 - fn: 12.0000 - accuracy: 0.9945 - precision: 0.9955 - recall: 0.9950 - auc: 0.9999 - prc: 0.9999 - val_loss: 0.3722 - val_tp: 1472.0000 - val_fp: 296.0000 - val_tn: 1925.0000 - val_fn: 108.0000 - val_accuracy: 0.8937 - val_precision: 0.8326 - val_recall: 0.9316 - val_auc: 0.9586 - val_prc: 0.9313 - 25s/epoch - 191ms/step

Now, if my labels are one-hot encoded, a softmax activation is used in the last layer for two classes and categorical crossentropy should be selected in model compilation:

x = Dense(2, activation="softmax")(x)
...
# Compile the model
model.compile(optimizer=Adam(), loss='categorical_crossentropy', metrics=metrics)
history = model.fit(X_train, y_train, 
                         batch_size=batch_size, epochs=number_of_epochs, 
                         verbose=verbosity_mode,
                         validation_data = (X_val, y_val),
                         callbacks=[es, cc])

But then my metrics are shown in a different way:

Epoch 59/60
132/132 - 25s - loss: 0.0217 - tp: 2408.0000 - fp: 11.0000 - tn: 2408.0000 - fn: 11.0000 - accuracy: 0.9945 - precision: 0.9945 - recall: 0.9945 - auc: 0.9999 - prc: 0.9999 - val_loss: 0.3722 - val_tp: 1472.0000 - val_fp: 296.0000 - val_tn: 1472.0000 - val_fn: 296.0000 - val_accuracy: 0.8937 - val_precision: 0.8937 - val_recall: 0.8937 - val_auc: 0.9586 - val_prc: 0.9313 - 25s/epoch - 191ms/step

It seems that true positives equal true negatives, and false positives equal false negatives, that is, all the metrics are strangely shown.

Does anyone know a solution to this? That is, a way to see metrics during training in the latter setup (one-hot, softmax, categorical crossentropy) as shown in the former setup (int, sigmoid, binary crossentropy).

Thank you!

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.