Definition of Type 1 and Type 2 Errors in Classification

Question

Why are the type 1 and type 2 errors as defined in bankruptcy prediction, different from type 1 and type 2 errors based on confusion matrix?

In bankruptcy literature:

Type 1 error: predicting a bankrupt company as a nonbankrupt one.

Type 2 error: predicting a nonbankrupt company as a bankrupt one.

In confusion matrix:

Type 1 error: predicting a negative case (nonbankrupt company) as a negative (bankrupt) one.

Type 2 error: predicting a positive case (bankrupt company) as a negative (nonbankrupt) one.

Thanks in advance.

Answer 1

What you said is true. Both articles look contradicting and arrive at opposite Type 1 and Type 2 errors.

I have had the same confusion several times.

Let me try to tell you what i concluded, though i am not 100% sure that my conclusion is right.

Definition of Type 1 error depends on False Positive. Definition of Type 2 error depends on False Negative.

Above two statements are always correct.

Then why there is confusion?

Confusion arises by the definition of Null Hypothesis and Alternate Hypothesis functions.

In statistics generally Null Hypothesis is the one that is mostly true and everybody accepts and doesn't contain much information and Alternate Hypothesis is the one that unsettles status quo and if true is some big news.

So, in your banking example you can consider

• Null Hypothesis = Bank is Normal (Solvent)

• Alternate Hypothesis = Bank is Bankrupt

So, if this is the definition, then bankruptcy literature is correct.

But, in general in Machine Learning/Data Science we don't define two hypothesis, we just say our problem definition as predicting if a bank is bankrupt or not. In this scenario, our

• True Positive is Bank is Bankrupt and we predicted it as Bankrupt.
• False Positive is Bank is Solvent and we predicted it as Bankrupt.
• False Negative is Bank is Bankrupt and we predicted it as Solvent.

So, again by the above statements, definition by confusion matrix is also right.

Then, why contradicting errors?? Answer is just the difference in the way we define hypothesis.

In banking literature, they defined true as solvent bank and false as bankrupt bank. In confusion matrix, we assumed true as bankrupt bank and false as solvent bank.

Have a look at the following for more detailed explanations.

https://www.khanacademy.org/math/ap-statistics/tests-significance-ap/error-probabilities-power/v/introduction-to-type-i-and-type-ii-errors

https://www.abtasty.com/blog/type-1-and-type-2-errors/