I am working on a multi-label classification NN using genomic data. there are 10 samples and 2 ground truth labels (age and gender) for every sample. I use a sigmoid activation at the final layer and since samples are no longer constrained to the probability distribution across classes, I get samples that might have a probability of 0.5 or above for multiple ages - say age1 and age2. Since this is genomic data, this is not surprising, especially with the distribution of ages.
I am generating confusion matrices as well as precision and recall reports using sklearn
from sklearn.metrics import classification_report, multilabel_confusion_matrix preds_train = model.predict(X_train) preds_test = model.predict(X_test) preds_train = preds_train > 0.5 preds_test = preds_test > 0.5 preds_train = tf.cast(preds_train, tf.float32)#one-hot predictions preds_test = tf.cast(preds_test, tf.float32) print(classification_report(y_true_train, preds_train)) print(classification_report(y_true_test, preds_test)) multilabel_confusion_matrix(targets_train, np.array(preds_train))
As mentioned, I get a scenario in which some samples have probabilities of more than 0.5 for the correct and also for the 'incorrect' class producing a one-hot vector containing three 1's for that sample as opposed to two. This means these samples get a turn at being true positive when evaluating precision-recall and F1 for that class as well as being a false positive for the incorrect class.
My question is, in principle is this ok and valid? Apart from increasing the threshold is there anything else that can be done?
I would be inclined to say that precision and recall and F1 are still correct, as it doesn't matter if a sample is TP and FP if this is just the pattern that is present and represents what future predictions might show!
I appreciate any input. Thank you!