Model/variable gini?

Question

I'm working with a colleague concurrently between R and MS Excel looking at credit risk scorecard modelling. In Excel he has calculated what he says is the gini coefficient for certain variables, which he has calculated by ranking the variable from lowest to highest, calculating the cumulative number of insolvencies, cumulative population, and using these to calculate a "width of the ranking" and ultimately the area explained by the variable.

The model is a simple logistic regression where I can add more variables or different variables depending on what people ask about.

mylogit <- glm(insolvency ~ LogPnL, data=my_data, family-"binomial")

However, in the Excel document the output from the model isn't used in the above calculations.

I researched how to calculate the gini coefficient in R and ended up calculating the AUC of a ROC curve like so:

# Full Model

predicted <- predict(mylogit, my_datafs, type="response")

#calculate AUC

aucc <- auc(my_datafs$Insolve,predicted)

gin <- 2*aucc-1
giin <- gin/(1-0.006059979) #where 0.006059979 is the insolvency rate
print(giin)

And this gives an entirely different number to what my colleague gets (for instance, I may get 0.6% whilst he gets 30%). I also tried a few other approaches:

library(WVPlots)

WVPlots::GainCurvePlot(my_datafs,"LROC","Insolve",title="Test Plot")

and

roc(my_datafs$Insolve ~ mylogit$fitted.values, plot=TRUE, legacy.axes = TRUE)

I seem to always get the same values using these approaches, but this is entirely different to what my colleague has calculated. So I asked him if this "gini coefficient" calculation has another name as when research it I only got the ROC and AUC stuff, and things about the Lorenz curve and economics. He suggested looking into gains tables/lift charts. I also looked into this and followed this site here but this does not work for me at all and just gives constant level values.

So my question is, does anyone know what my colleague is calculating and how I can do this in R and verify what has been done?

The data looks something like this (where 1s represent insolvency in column a):

Insolvency  LogPnL  LogAssets   LogReturnoncapital
0   13.45244524 17.26029721 -4.555781778
0   -13.16158409    17.26053342 -0.610391211
0   15.33151653 17.26059723 -4.62544939
0   15.24483998 17.26060402 -1.08183692
0   -12.40954396    17.26068645 -3.763048412
0   15.17672144 17.26070709 -1.438018097
0   15.16098292 17.26075672 -1.438018097
0   15.21341303 17.26084054 -4.852438172
0   15.62576461 17.26085241 -1.911767818
0   15.13992952 17.26094809 -2.296309704
0   15.1798149  17.26094809 -0.742112526
0   15.94790027 17.26094809 -1.719503458
0   15.44470345 17.26105944 -0.890755178
1   -15.53863423    17.26107564 -0.779659645
1   14.64142528 17.26116973 -2.536352638
0   -14.06471164    17.2611713  -4.707113261
0   15.37648401 17.26119409 -1.812813986
0   15.43226742 17.26123242 -1.245680522
0   14.11857373 17.26123506 -3.67956894
1   14.25847374 17.26129203 -22.89380415
0   -14.48845503    17.26129882 -0.3949376
0   13.635187   17.26129882 -4.97512426
0   14.88228812 17.26129882 -1.299654895
0   13.46595308 17.26136258 -4.948858859
0   15.6823775  17.26142633 -0.976068273
0   12.80490915 17.26145821 -2.103263152
0   14.80132735 17.26149008 -6.06110278
0   14.94400522 17.26152196 -2.778127905
0   15.07907215 17.26152196 -6.098750561

Answer 1

Unfortunately, there are multiple measures called gini coefficients or gini index and they are used for different things in different domains. So you are for sure not the first person to face this problem.

Luckily, for your domain it is clear what coefficient to use.

Gini coefficient for (credit risk) scorecards

The gini coefficient to evaluate the predictive power of a credit risk scorecard is given by $$gini = 2*rocauc-1$$ This is (one of) the standard measure for evaluating credit risk scorecards, so this should be the one your colleague is calculating.

So for your code, just remove the line giin <- gin/(1-0.006059979) from your code and use gin and you should be fine.

Disclaimer: I am no expert for R, but if the code does, what it implies, then the change should be enough.

If your colleagues values that strongly from yours, he probably does not compute the gini coefficient that is common for credit risk scorecards.

Some Background

Both, roc-auc and gini are measures that evaluate the order of the scores, not their actual value. So there should be no difference whether you use the linear term of the logistic regression as score or the logistic mapping into probabilities.

The roc-auc can be interpreted as the probability that a random insolvency case gets a higher risk score than a random non-insolvency case. This means that

• a perfect model, where all insolvency cases get a higher score, gets a roc-auc of 100%.
• a model that assigns random scores independent of any feature or information gets a roc-auc of 50%.
• a model that often estimates the risk of insolvency cases below the one of non-insolvency cases might be below 50%.

Similarly, the gini assumes that all models are better than random scores. Hence the baseline of a random score is set to 0% gini and a perfect model gets a gini of 100%. If it happens that the gini is below 0, swapping the scores (e.g. new_score = 1 - old_score) will remove the negative sign of the gini.