Custom Metrics with Keras

Question

How to define a custom performance metric in Keras?

I am trying to use it but I can not see the metrics values on each epoch.

clf51.compile(optimizer=sgd51, loss='binary_crossentropy', metrics=["accuracy"])

clf51.fit(X_train, Y_train, batch_size=384, epochs=5, callbacks=[metrics], validation_split=0.30, verbose=2)

As you can see:

Train on 139554 samples, validate on 59810 samples

Epoch 1/5 - 4s - loss: 0.3576 - acc: 0.9885 - val_loss: 0.0531 - val_acc: 0.9989

Epoch 2/5 - 3s - loss: 0.0261 - acc: 0.9987 - val_loss: 0.0135 - val_acc: 0.9987

Am I doing something wrong? I would like to make a earlystop using f1s.

Answer 1

The callback you are using isn't for displaying the desired metrics, just recording them. For example if you want to access the F1-score you need to type: metrics.f1s. This is useful, let's say, if you want to make a graph on how the F1-score reduced during training.

To use the EarlyStopping callback, however, f1-score needs to be a metric not a callback like you have it!

You need to write (or find) a function that calculates the F1-score through keras' backend functions. You might want to check if this works for you.