3
$\begingroup$

For some $8000$ customer profiles, in addition to a data-set, I have two kinds of scores available:

  • Type 1 Score ranges from $0$ to $1$ and gives the prediction probability of that profile belonging to a class. The distribution of this score is severely skewed, as shown

enter image description here

  • Type 2 Score ranges from $-5$ to $5$ and is a risk score for the profile obtained using the PRIDIT method on the actual data. The distribution of this score is fairly normal

enter image description here

The objective is to bin the profiles into some $N$ bins using the aforementioned information. The bins should represent various degrees of risk associated with that profile.

My first thought was to cluster the profiles based on the two scores available to us. However no meaningful clusters were observed.

The next idea was to sort the data on Type 1 Score and then cut on the points where there's a sudden change in slope, which seemed like a good idea at first but here's what the plot for Type 1 Score looks like ...

enter image description here

What I am thinking now is to transform Type 1 Score and Type 2 Score together to give some pseudo-normal type of distribution, hoping that some kind of discretization might give the correct bins.

My questions are:

  • How can I transform the two scores into a pseudo-normal distribution?
  • What would be a better way to approach the above problem?
$\endgroup$
2
$\begingroup$

It often is appropriate to use quantile bins.

I.e. to get four bins, choose the smallest 25%, then up to the median, then the Q75, ...

| improve this answer | |
$\endgroup$
  • $\begingroup$ Are quartiles really a good representation of degrees of risk with the kind of distribution seen? $\endgroup$ – 119631 Jul 26 '16 at 13:52
  • 1
    $\begingroup$ Yes, quantiles also work on skewed data. $\endgroup$ – Has QUIT--Anony-Mousse Jul 26 '16 at 19:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.