Let's say I have model A and model B, that I want to compare the performance of by inspecting a confusion matrix. They both produce a list of predictions,
pred_B, corresponding to a set list of ground truth values
ground_truths. I can use something like
sklearn.metrics.confusion_matrix in order to generate a confusion matrix for each of these models:
cm_A = confusion_matrix(ground_truths, pred_A) # e.g. [[5,1], [2,3]] cm_B = confusion_matrix(ground_truths, pred_B) # e.g. [[6,0], [3,2]]
What I want to figure out is how to get the specific indices from each cell of the confusion matrix, corresponding to the ground truth class. This will let me investigate which specific data points are true-positives, false-positives, false negatives and true negatives, and let me search for patterns based on their input features. In the example above, are two of the false negatives the same false negatives in
cm_B? Or did all of the true positives switch with all of the false negatives between the two model predictions? If I knew their indices in the
ground_truths list I could answer the question easily.
I can think of stupid/inefficient ways to do this, but I would love to find a solution that could scale to an arbitrary number of labels.