# Identifying similar data points

I have a data set where customers who are non-delinquent with us have defaulted with others. So from the total data set, 30% are those who defaulted with others but 70% are those who are good with us and other others as well.

With the help of 30% customers who defaulted with others (being good/non-delinquent with us), can we identify the similar customers (matching in profile) from the remaining 70% customers who are currently good with us as well as with others, and also to any new customer?

This is required so that we can have an early indication that who are the customers (from remaining 70% & any new addition) most likely turn bad in near future?

I am not sure how to apply any machine learning algorithm (as we just have 1 class defaulted with others) to identify similar customers.

• Yes, you can. What have you tried so far ? – lcrmorin Jan 9 '20 at 9:37
• I am in planning phase and trying to finalize the approach. i was thinking some sort of clustering might work here but then i would have any control on data points and they may be put in different clusters. Another approach i though is to do classification with all data point with default(30%) and pick remaining 70% for other class. But challenge is some form 70% may turn default after some time. – SKB Jan 9 '20 at 9:56
• Any other feature from the customers aside from default with others? – Yohanes Alfredo Jan 9 '20 at 12:55
• all the features are matching, default with others is the additional one. – SKB Jan 9 '20 at 13:20

I think clustering is the way to go, but first you have to preprocess data in the proper way.

I would start encoding your observations using an Autoencoder for dimensionality reduction. This DL technique allows you to compress your data, or to take a "summary of it". The compressed representation of your data is often use as a preliminary step in outlier detection tasks, and I think your problem is highly compatible with this approach. If you don't want to use Deep Learning, you can recur to other dimensionality reduction techniques. PCA might work, for example. (t-SNE is stochastic, and I don't recommend it outside data visualization in lower dimensions.)

Once you have represented your data in a dense, lower dimensional space, there's a number of things you can do.

1. Employ some measure of distance between observations in this compressed space. If your goal is to understand how different two observations are, that would be the main thing to do. The most common and simple is Euclidean distance, but there's an awful lot of measures you can use (Manhattan, Minkowski, Mahalanobis, ... you name it!).

2. You can run clustering techniques. The one I use almost any time is, of course, k-Means clustering. With this, you can try to identify groups of more similar observations, this can give you hints on how to spot "good" and "bad" observations. An alternative is DBSCAN clsutering, that allows you to classify some observations as outliers (the drawback of this model is that you have many hyperparameters to tweak.)

3. Additionally, you can train a classifier that is fed with the compressed representation of your data. Labeling observations of a dataset as "good" and "bad" could be time consuming, but it might be worthwile if you don't just want to check how different two customers are, but also get a straight classification.

• Thanks Leevo, 1) I am still not sure how clustering will help here as there are high chances that the bad customers(my target class) may spread across many clusters. So i will not get the similar data points. 2) Classification : I am not sure which points to be classified good(apart from 30% which may turn bad in future). – SKB Jan 9 '20 at 10:40
• 1) Clustering might be useful, I can't say it will, since I don't have your dataset to inspect. 2) You wrote you need: "an early indication that who are the customers [...] most likely turn bad in near future". This is a classification problem. If you want your model to do that, you must label data at some point. It's a binary classification problem. – Leevo Jan 9 '20 at 10:46
• Agree Leevo, that to make it a classification problem we have to label the data but question is how? I also agree that clustering might help here but at the same time clusters have to be defined in such a way that they just pickup the customers who are similar to bad customers only in 1 cluster and good to be clustered in other cluster.. Not sure how? – SKB Jan 9 '20 at 10:52
• Clustering allows to simplify categorization of your observations in an unsupervised way. You can do much mora than that with it. For example, you could find that your observations tend to be clustered together in, say, k main groups. You could use this information to train your future classifier. This is only an example of course, I'm not saying it's what you'll need... let's see! There's also a value in the simple exploration of your dataset, sometimes you find useful insights that will let you tune a good ML model. Of course results don't come out by themselves simply running a k-Means. – Leevo Jan 9 '20 at 10:56
• Clustering may be usefull, notably for feature engineering, but it won't be the part of the algo that learn things in the context of credit scoring. – lcrmorin Jan 9 '20 at 14:00

First, I don't think this problem can be addressed as a binary classification problem. Indeed, your current label (customer has or has not defaulted with others) is not reliable, as some customers who have not defaulted in the past might do so in the future, with you or others.

Consequently, what comes to me as the first thing to try is one-class SVM (see the scikit-learn implementation), because you can only rely on a single class (the 30% of your dataset which correspond to the default class). OCSVM is useful when you have only one class in your dataset, and also popular in problems where the classes are so imbalanced that some are likely to have too simple patterns.

Theoretically, training an OCSVM over your default class and applying it to the other 70% will let you see which currently reliable customers are likely to default in the future. The distance to the separating hyperplane will even enable you to sort customers regarding their potential to default.

However, in practice, it won't be an easy task: you will especially find it hard to choose a kernel and its hyperparameters without a proper validation dataset, including a certain number of customers who will not default. To prepare this validation dataset, I trust that you can collect some past data of both customers who defaulted and not. As soon as you can do this, find the best kernel / hyper-parameters with a classical validation approach, trying to lower the false negatives rate (use the recall score as your main guide).

Please note that your problem is such that you will thus build a model to predict the risk of default for other customers. You might want to include data about customers who defaulted with your own company, and see if it changes the results.

It's generally treated as a binary classification problem, often called credit scoring. You are trying to know whether a specific entity will default with you or not. The fact that the entity has defaulted with another company can be used as feature of your problem. As mentionned by others, there are other approaches, but I honestly think they are in theoretical infency, and can't seem to find any literrature on their practical implementation to credit scoring.

Then you need to build an historical database to learn on, with an associated label. There are two main approaches : either you construct a table based on periodic (yearly) data and set an horizon for the default. It allows you to build a label by observing what happened in the past. Say you have yearly data and consider a 1 year horizon. You will have multiple line for a company : one for each year, say 2010 to 2018, the label will be built on their status the following year, 1 if they defaulted respectively in 2011, ..., 2019, 0 otherwise. Another approach is to consider life-time default, it is better in theory, and probably better match what you want to do, but it is unlikely you will be able to do that, notably because it require complete data and you usually don't keep those that defaulted.

You can learn on that database, but this is a complex process including variable selection, model selection, model training, model validation. Pretty much standard data science, there is a lot of introductory material you should look at, starting with : an introduction to statistical learning, which is free). Then you can go look at ressources oriented towards credit scoring like Credit Scoring and its application, then to supervised learning applied to credit scoring like deep learning for mortgage risk: https://arxiv.org/abs/1607.02470). This will help you measure the risk associated with your current population. More so if you calibrate your model in probability such that the output of your model reflect the probability of default.

However if you want to use that model for making decisions about lending, then the problem will be made more complex, because :

1) Your initial decision will have an impact on the credit situation of said entity. So you actually would need to assess the impact of both legs of your decision on the caracteristic of the entity before putting it in your prediction model.

2) You need to take costs into account. The costs are usually asymetrical and hard to predict. For loans for exemple, not doing the loan will cost you interests on the nominal (it loops to (1) as the % interest you may ask will modify the contract and hence the situation of the entity), but also associated returns (a bit of money for maintaining their account for exemple), while a default would cost you a proportion of the outstanding amount, which is another order of magnitude, and that proportion is often rather difficult to predict (given insurances for exemple).

I am not really aware for advanced literature for points 1) and 2), despite working in the field, I even suspect there is a complete black-out on those topics as they are the one that provide competitive advantages to a given credit lender.

3) You want to avoid social discrimination, mostly for ethic and legal purpose, which is counterproductive to statistical discrimination. There is not much literature on the topic in general, and fewer in credit scoring, except maybe : https://arxiv.org/abs/1610.02413. Depending on your jursidiction this may give you some headaches.

It's a very broad task and there is no single answer. You have to experiment. Look for active learning-based approach. Also Single Class SV<

Saurabh without the data this is a little difficult to answer but here goes

I would agree to Leevo's thought process here since 30% of your data is folks who have defaulted elsewhere and you want to use that to understand who from the remaining 70% and anyone new would default in the future. Clustering would be the best approach here and since the trend of the data plays an important part usage of Mahalanobis distance over euclidean distance might give a good cluster consistency if you try k-means clustering here. Another alternative is to try using association rule mining to understand what variable combination contribute to customers defaulting at others and then use those rules that association rule mining throws out on the new data set.