1
$\begingroup$

I have a data like as shown below (working on classification problem using traditional classification and DL based approaches)

enter image description here

I see in feature engineering tutorials (and tools) here and here, they usually compute basic statistics features based on numeric column such as max(loan amount), min(loan amount), sum(loan amount),stddev(loan amount), average (loan amount) etc.

I understand all these are done in an attempt to increase the predictive power of the model.

However, my question is

what does it mean when max(loan amount) or std dev(loan amount) is an important feature? can help me understand what insight does it convey? how to interpret this feature? can explain in simple english?

Let's assume we run a random forest model and in the feature importance we see that max(loan amount) is the top most feature. What does it mean? I am looking for meaning to understand the insight that it communicates. This question is not about the model. It's simply about the meaning of the term/feature std dev (loan amount) or max(loan amount) or min(loan amount)

$\endgroup$

2 Answers 2

0
$\begingroup$

First it's important to define the problem: let's assume that the goal is to predict for a person identified by their s_id the probability that they would default on a particular loan (or the risk that they would default in general).

So in this setting one instance represents one person. The available data contains information about the person's past loans. First there's a technical issue: this history can be any length, but with traditional feature representation we need a fixed length vector of features. Semantically, it's a matter of providing the model with the information contained in this history in a way which is useful to predict the target variable: intuitively, the exact amount of every loan is not essential since it's very specific (it might even even cause some overfitting if used directly).

For these reasons it makes sense to "summarize" the loan history as a fixed-size vector of stats. Typically number, mean, median, possibly quantiles, standard deviation, etc. These values make the instances comparable to each other, in the sense that the model can distinguish customers with different patterns in their history. In this particular case it would certainly make sense to create two series of stats: one for the loans paid and the other for the unpaid loans, since this is clearly something which can help the model.

$\endgroup$
9
  • $\begingroup$ @ Erwan - thanks for the help. Upvoted. Useful. My questiom is mainly to know what does 'max(loan_amount)' mean? How to interpret this term? For ex: let's say that salary has a high negative coefficient with loan default, then we can interpret that as salary increases, likelihood of loan default decreases. In other wprds, people who earn more, default less. Or people who earn less are more likely to default. Similalry, can you help explain in simple english, what isbthe meanimg of 'max(loan amount)' ? $\endgroup$
    – The Great
    Commented Feb 25, 2022 at 23:27
  • $\begingroup$ This is question is mainly on understand the meaning of 'max(loan amount)', 'min(loan amount') in linguistic sense. How would you explain to a business user, when you see 'max(loan amount') as a important feature with an example. I am aware that it helps to get pure split. But what does 'max(loan split)', how to interpret this term/feature itself (before we go to model etc). $\endgroup$
    – The Great
    Commented Feb 25, 2022 at 23:33
  • $\begingroup$ @TheGreat ok apparently I didn't understand your question, but imho you're overthinking this: max(loan_amount) is simply the highest loan amount a customer had, and this is an indication of which amount this customer can be trusted with: if a customer previously had a loan of 100,000, then it means that previously the bank estimated that the customer can repay this amount. Whereas if the max loan is 2000, it means that this customer never had larger loans and therefore the bank should be careful if they apply for a 50,000 loan. $\endgroup$
    – Erwan
    Commented Feb 26, 2022 at 11:29
  • $\begingroup$ Thanks, this was the explanation I was looking for. So, let's say if customer's max(loan_amount) earlier was 10K USD but he defaulted on that loan, then bank should be careful about offerring another loan for similar value. Am I right to understand this? We need to interpret this feature along with the outcome (default/not default). Am I right? Just knowing what was his previous/earlier max(loan_amount) would not help. Am I right? $\endgroup$
    – The Great
    Commented Feb 26, 2022 at 14:30
  • 1
    $\begingroup$ @TheGreat yes absolutely, you could have two features: one for the max loan repaid, one of the max with default if any. About std dev: it quantifies the spread of the distribution, so it means that a low std dev indicates that the customer has a pattern of asking around the same amount every time, which may be a sign of good budgeting behaviour if the loans are repaid. By contrast a high std dev indicates irregular loan amounts, so approval may require more caution. $\endgroup$
    – Erwan
    Commented Feb 26, 2022 at 15:36
1
$\begingroup$

When max(loan amount) is the most important feature in a random forest classification, it is most often the first feature to make the decision split. In other words, that feature is the most useful in dividing the data in homogeneous classes.

$\endgroup$
2
  • $\begingroup$ @ Brian Spiering - thanks for the help. Upvoted. Useful. My questiom is mainly to know what does 'max(loan_amount)' mean? How to interpret this term? For ex: let's say that salary has a high negative coefficient with loan default, then we can interpret that as salary increases, likelihood of loan default decreases. In other wprds, people who earn more, default less. Or people who earn less are more likely to default. Similalry, can you help explain in simple english, what is the meanimg of 'max(loan amount)' ? $\endgroup$
    – The Great
    Commented Feb 25, 2022 at 23:28
  • $\begingroup$ This is question is mainly on understand the meaning of 'max(loan amount)', 'min(loan amount') in linguistic sense. How would you explain to a business user, when you see 'max(loan amount') as a important feature with an example. I am aware that it helps to get pure split. But what does 'max(loan split)', how to interpret this term/feature itself (before we go to model etc). $\endgroup$
    – The Great
    Commented Feb 25, 2022 at 23:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.