0
$\begingroup$

I have a set of independent variables X and set of values of dependent variable Y. The task at hand is binary classification, i.e. predict whether debtor will default on his debt (1) or not (0). After filtering out statistically insignificant variables and variables that bring about multicollinearity I am left with following summary of logistic regression model:

Accuracy ~0.87
Confusion matrix [[1038 254]
                 [72 1182]]
Parameters Coefficients
intercept  -4.210
A          5.119
B          0.873
C          -1.414
D          3.757

Now, I convert these coefficients into new continuous variable "default_probability" via log odds_ratio, i.e.

import math
e = math.e
power = (-4.210*1) + (A*5.119) + (B*0.873) + (C*-1.414) + (D*3.757)
default_probability = (e**power)/(1+(e**power))enter code here

When I divide my original dataset into quartiles according to this new continuos variable "default_probability", then:

1st quartile contains 65% of defaulted debts (577 out of 884 incidents
2nd quartile contains 23% of defaulted debts (206 out of 884 incidents)
3rd quartile contains 9% of defaulted debts (77 out of 884 incidents)
4th quartile contains 3% of defaulted debts (24 out of 884 incidents)

At the same time:

overall quantity of debtors in 1st quartile - 1145
overall quantity of debtors in 1st quartile - 516
overall quantity of debtors in 1st quartile - 255
overall quantity of debtors in 1st quartile - 3043

I wanted to use "default probability" to surgically remove the most problematic credits by imposing the business-rule "no credit to the 1st quartile", but now I wonder whether it is "surgical" at all (by this rule I will lose (1145 - 577 = 568 "good" clients) and overall is it mathematically/logically correct to derive new continuous variables for the dataset out of the coefficients of logistic regression by the line of reasoning described above?

$\endgroup$
1
$\begingroup$

Deriving a probability value (the continuous variable you are talking about) from the logistic model is a perfectly sound thing to do. The probability value is actually the main output from the model.

Getting from the probability value to a decision rule (e.g. from default probability to credit granting decision) is an another step that will also need to incorporate a number of business decisions concerning risk appetite - i.e. how risky clients are you willing to approve not to miss a potentially large amount of business, what the interest is going to be for different risk scores etc. The tradeoff this refers to is the well known sensitivity-specificity tradeoff which is, apart from the confusion matrix you are using, probably best visualised by the ROC curve.

From the confusion matrix it is also apparent that you were training the model on a balanced sample (default rate around 50%). This is quite unusual in credit risk modeling, usually the default rate is well below that. If that is the case, you will probably need to calibrate the probabilities, for example by fitting a yet another logistic regression model on your probabilities as a single variable (this works best when the probabilities are normally distributed). The calibrated probabilities that this second model will predict will accurately reflect the actual real-world probabilities of default.

$\endgroup$
2
  • $\begingroup$ thank you for valuable comments. regarding the balance of sample: I used SMOTE algorithm to generate addiitonal sample for minority group (original distribution is 85% regular payers vs. 15% of defaulted ones). Did I screw up by implementing SMOTE? $\endgroup$ Mar 7 '19 at 17:07
  • 1
    $\begingroup$ No, SMOTE is a good way to obtain a balanced sample. However, the probabilities from such a model trained on a balanced sample will average around 50%, which is not the actual probability of default - therefore, calibration will be needed to obtain the actual probabilities as described above. $\endgroup$ Mar 9 '19 at 9:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.