Which Keras metric for multiclass classification

Question

I have a multiclass classification data where the target has 11 classes. I am trying to build a Neural Net using Keras. I am using softmax as activation function and categorical_crossentropy as the loss function. I have one hot encoded the target before passing it into the net. The issue I am facing is which Keras metric should I use for this purpose? The official documentation does not mention which metric is suitable for multiclass classification.

This link mentions to use categorical_accuracy as the metric for multiclass classification but other than that, all other question on this site are about multilabel classification metrics like this and this link.

Is there any implementation of lets say f1_score in Keras using the custom metric function, since f1_score is the go to metric for multiclass classification I guess?

EDIT 1:

Would something like this work using the custom metric functionality in Keras?

from sklearn.metrics import f1_score

def my_metric_fn(y_true, y_pred):
    f1 = f1_score(y_true, y_pred)
    return f1

model = Sequential()

model.add(Dense(input_dim = 12, units = 128, activation = 'relu', kernel_initializer = 'he_uniform'))
model.add(Dense(units = 64, activation = 'relu', kernel_initializer = 'he_uniform'))
model.add(Dense(units = 32, activation = 'relu', kernel_initializer = 'he_uniform'))
model.add(Dense(units = 11,  activation = 'softmax', kernel_initializer = 'glorot_uniform'))
model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = [my_metric_fn])
history = model.fit(train_x5, train_encoded, validation_split = 0.2, epochs = 5, batch_size = 1000)

EDIT 2: This is giving me an error in the last line as follows: OperatorNotAllowedInGraphError: using a tf.Tensor as a Python bool is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.

Answer 1

One option is to implement F1 score in Keras:

from tensorflow.keras import backend as K

def f1(y_true, y_pred):    
    def recall_m(y_true, y_pred):
        TP = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
        Positives = K.sum(K.round(K.clip(y_true, 0, 1)))
        
        recall = TP / (Positives+K.epsilon())    
        return recall 
    
    
    def precision_m(y_true, y_pred):
        TP = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
        Pred_Positives = K.sum(K.round(K.clip(y_pred, 0, 1)))
    
        precision = TP / (Pred_Positives+K.epsilon())
        return precision 
    
    precision, recall = precision_m(y_true, y_pred), recall_m(y_true, y_pred)
    
    return 2*((precision*recall)/(precision+recall+K.epsilon()))