3
$\begingroup$

If we have a loan book and want to train the data to predict the probability of default, what is an appropriate way to sample the historical data to train the model, given that each account is open over a period of time and not just at a single snapshot?

For example - for predicting defaults of new customers on a loan book, it is easy to select data to train a model at an equivalent point in time - i.e. when each customer is new, we may look at the FICO score, age of customer, home State etc at the point of loan application.

But once a loan has been made and we wish to build a model for likelihood of default given ongoing repayment history, what is the statistically appropriate sample to take? A random sample within the life to date of the loan picking each record at a single random point in time, choosing all loans at a specified point in time (e.g. at exactly 6 months in from origination) or a sample of dates covering a period?

Or does it require something else - is a classification model actually appropriate for this type of scenario (assuming a binary outcome), or should some sort of time series or survival model be considered?

Improve this question
$\endgroup$
2
  • $\begingroup$ As from what I understand from your dataset you have a temporal dependency and you want to predict new incoming customers, right? $\endgroup$
    – Carlos Mougan
    Feb 12, 2020 at 11:21
  • $\begingroup$ It's more of a churn type model. You have existing customers who make payments back against their loans on a regular basis (once per month for example). I am trying to predict which customers will stop making payments (default) at some point in the future (say the next 3 months). You would expect that someone who has paid back 90% of the loan and therefore has a proven payment history is less likely to default than someone who has just taken out the loan (for example). But when training the model, how best to account for the temporal nature of the data? $\endgroup$
    – Brisbane Pom
    Feb 12, 2020 at 22:09

2 Answers 2

Reset to default
1
$\begingroup$

You should have a look at this Kaggle competition on fraud detection. In this kernel the winning team explains their approach.

This could be a time series approach (same as yours) but in this case:

We are not predicting fraudulent transactions. According to the competition host Lynn here. Once a client (credit card) has fraud, their entire account is converted to isFraud=1. Therefore we are predicting fraudulent clients (credit cards).

For this team that won the challenge the solution was:

We did a CV GroupKFold using month as the group.

It seems that perhaps you could use the same validation strategy.

https://www.kaggle.com/c/ieee-fraud-detection/discussion/111284

Improve this answer
$\endgroup$
0
$\begingroup$

There are two main approaches : either you consider the whole life of the loans or chunks of it. These two approaches reflect the two underlying problems of credit scoring : granting the loan and measuring the risk associated with your portfolio.

Looking at the whole life of the loan :

  • Your features will be the situation of the customer at the start of the loan.

  • Your target will be the occurence of a default event during the life of the loan.

  • Pro : it will help you make the decision to grant the loan or not.

  • Cons : it will be difficult to gather data (one repaid/defaulted loan = one instance) to learn on, it will be difficult to deal with current loans if the situation of the customer has significantly evolved.

Considering chunks of it :

  • You define a periodicity and an horizon

  • Your feature are the situation of the customer at the start of the period

  • Your target is the occurence of a default (or not) after the horizon period

  • This will help measure the risk associated with your portfolio up to the horizon (but won't help granting a loan or not)

  • This will allow you to get more data and deal with relatively recent loans.

Improve this answer
$\endgroup$
3
  • $\begingroup$ OK - so I would want to exclude your first scenario as I am comfortable with that approach - there is after all only one point in time at which each account is at the start of the loan. I am interested in the second scenario - how to measure the future risk of default for existing loans. Your approach suggests looking at customers at the start of a defined period - again a single snapshot in time - which would give a range of account vintages. But is there an approach that could be used that re-samples the same accounts at different periods or points in time? $\endgroup$
    – Brisbane Pom
    Feb 14, 2020 at 7:08
  • $\begingroup$ I am not sure to understand your question. In my exemple, any given account will be sampled different time (at each year start in my exemple). $\endgroup$
    – Lucas Morin
    Feb 14, 2020 at 12:59
  • $\begingroup$ OK - so we take (for example) all accounts that were active on 1 January, look at their attributes at that time (age of loan, number of prior missed payments etc) and decide if they go into default or not in the next 3 months. Then we repeat the exercise on 1 April and again on 1 July etc, so each account that was active on these dates is effectively re-sampled. Is this legitimate? Can we capture the resample dates at different periods (e.g. each week)? Or should we look to use a model that accounts for time in a different way? $\endgroup$
    – Brisbane Pom
    Feb 15, 2020 at 21:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.