# Confusion matrix, when mistakes below diagonal are better than above the diagonal

I have a classification problem and I am producing a confusion matrix. Ideally one wants to get all results in the diagonal. I get quite many points around diagonal for different algorithms. Still for my use-case I want to favor algorithms that underpredict the class (I have ordinal data) and not overpredict.

Is there a metric that can measure under and overprediction and rate those errors with a different weight? The typical accuracy, precision terms assume that all mistakes are the same.

Of course I can try to implement my own metric but I am quite sure that I am not the first that is having this issue.

Any metric available that you know already? Thanks Alex

## 1 Answer

You are incorrect when you say accuracy and precision assume the same mistakes. They are quite different in nature as to when they are applied for separate use-cases.

For more details on how they are different - https://towardsdatascience.com/accuracy-precision-recall-or-f1-331fb37c5cb9

As per your situation, you could opt for precision as you are looking only for the right answers, in case of a spam email detection.

There is no metric as such which calculates how much your model underfits or overfits but a genera idea can be made when you compare the validation curve and the training curve.

• My bad phrasing. Accuracy and prediction do not differentiate on the type of error. Also I am not sure I want to see if my model overfits or underfits but rather if the model has some tendency (i.e more frequently to predict a class higher of what it is) (i.e more rarely predicts a class lower of what it is) – Alex P Jan 6 at 8:43
• Could you elaborate on "class higher" and "class lower"? – Aymuos Jan 7 at 1:20
• sure! Lets say that you have the following classes that refer to transmission speeds. Class 1,2,3,4,5,6. Class 1 is the lowest speed and Class 6 is the highest. Assume that we wanted to predict class 3 but there are two algorithms. Algorithm A: Predicts 2 (1 less) and Algorithm B: predicts 4 ( 1 more). Accuracy and precision will count those errors equally (u have just missed the right class). For a real use-case algorithm A is more useful (shall I elaborate the why?) . Algorithm B at the other side (predicting more) can have quite devastating results. – Alex P Jan 11 at 8:07