I ran a logit model using statsmodel api available in Python. I have few questions on how to make sense of these

1) What's the difference between summary and summary2 output?

2) Why is the AIC and BIC score in the range of 2k-3k? I read online that lower values of AIC and BIC indicates good model. Is my model doing good? Is there any optimal range for AIC and BIC?

3) As you can see covariance Type is non-robust. What is it and should I be concerned about it?

4) Is there any other field/item in the output that I should pay attention to?

5) You can see below how certain significant variables like X2,X8,X45 have very low coefficients. How can they be significant and still have very low or near to zero coefficient? Is it normal?

This is the output that I got

Summary output

Dep. Variable:  vae_flag        No. Observations:   3298
Model:  Logit                   Df Residuals:   3241
Method: MLE                     Df Model:   56
Date:   Mon, 30 Dec 2019        Pseudo R-squ.:  0.3347
Time:   21:18:36                Log-Likelihood: -1392.2
converged:  True                LL-Null:    -2092.7
Covariance Type: nonrobust      LLR p-value:    3.894e-256

Summary2 output

Model:  Logit                   Pseudo R-squared:   0.335
Dependent Variable: op_flag     AIC:    2898.4259
Date:   2019-12-30 21:18        BIC:    3246.1870
No. Observations:   3298        Log-Likelihood: -1392.2
Df Model:   56                  LL-Null:    -2092.7
Df Residuals:   3241            LLR p-value:    3.8937e-256
Converged:  1.0000              Scale:  1.0000
No. Iterations: 7.0000  

Significant variables

      coef  std err   z     P>|z|   [0.025  0.975]
x2   0.0321 0.060   11.227  0.000   0.558    0.794
x6   2.2996 0.095   24.332  0.000   2.114    2.485
x7  -1.8795 0.082   -22.835 0.000   -2.041  -1.718
x8   0.0002 0.058   2.116   0.034   0.009    0.237
x16  0.2693 0.059   4.564   0.000   0.154    0.385
x33 -0.3138 0.139   -2.254  0.024   -0.587  -0.041
x34  0.4644 0.137   3.392   0.001   0.196    0.733
x45  0.0088 0.052   2.306   0.021   0.018    0.221
x52 -0.1755 0.087   -2.007  0.045   -0.347  -0.004
x55 -0.0982 0.050   -1.965  0.049   -0.196  -0.000

Very brief answer (no way to go into details here):

  1. Aparently the two calls produce different tables containing slightly different statistics.
  2. AIC and BIC compare nested models. So if you have some model and you add or remove some variables (for instance), you may compare AIC, BIC. There is no universal "okay" range in terms of overall figures. Even with a low(er) AIC, BIC, you can have a "bad" model. So AIC, BIC really is about comparing "similar" models against each other.
  3. There are robust standard errors, which are computed in a different way than "normal" standard errors. I think this indicates that "normal" standard errors are calculated.
  4. Not really. The pseudo R-squ. may give you some idea about model fit, but it is a little different from "normal" R-squared and I don't find it too useful.
  5. The value of the coefficient is not directly related to significance (small values can be significant and vice versa). Significance is calculated from standard errors/t-/z-statistics. Note that your coefficients are log-odds (NOT marginal effects). In case you want to obtain marginal effects, you need to look for some package (like "margins" in R/Stata) or you do this by hand.

Overall I recommend to have a good read about logistic regression since you seem to be uncertain about basic concepts.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.