# Getting value of 2 in normalized confusion matrix

I have created confusion matrix with python and I have gotten weird results that made me worried:

as you can see, I have value of two, and if I understand correct, normalized matrix cannot have value higher than 1. and also, some columns' sum is not 1.

this is how I created this matrix:

df_confusion = pd.crosstab(df['labels'], df['prediction'])
df_conf_norm = df_confusion / df_confusion.sum(axis=1)


and then I just plot it.

my question is - is it possible to get value of 2? does the sum of column make sense not to be 1? do I have mistake? and if correct, what can explain that?

EDIT

• Please print a few rows of - labels and prediction. – 10xAI Jul 9 at 13:45
• @10xAI I have jsut added – Reut Jul 9 at 18:33

Switch the axes:

# Build the table
confusion_table = pd.crosstab(df["prediction"], df["labels"])

# Normalise
confusion_table = (confusion_table / consufion_table.sum(0))

# Verify
print(confusion_table.sum())


Alternatively, you can play with confusion_table.divide and the axis argument: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.divide.html

• Thankyou for your answer, I did it and now the sum is ~1, like I get 1.01 or 0.99 , do you know why that can happen? – Reut Jul 12 at 9:30
• Mathematically, it should not happen. What can happen is that you have small differences because of the floating point but it's more of the order of 1e-9 than 1e-2. You can also use the function confusion_matrix provided by sklearn.metrics, that way you are certain that it's correct – qmeeus Jul 12 at 19:56