2
$\begingroup$

I have created a model using Logistic regression with 21 features, most of which is binary. I created these features using get_dummies. Few of the other features are numeric. I get a very good accuracy rate when using a test set. On checking the coefficients, I am not able to interpret the results. I understand that the coefficients is a multiplier of the value of the feature, however I want to know which feature is most significant. How do I interpret by the coefficient value?

Coefficients are:

[[9.94670383e-01 1.11134574e+00 1.52026366e+00 1.00438802e+00
  1.00526047e+00 9.99706283e-01 1.16615129e+00 7.62542057e-01
  2.07560050e-01 6.37260544e+00 5.04481410e+01 1.25174912e+00
  1.41552939e-02 1.00212609e+00 8.99670711e-01 1.54314635e+00
  1.40966824e+00 2.59413851e+00 9.60362518e-01 7.12430063e-01
  9.49015749e-01 8.73217396e-01 1.13360003e+00 1.29499497e+00
  2.59413851e+00 8.48199127e-01 1.29459832e+00 8.39932559e-01
  6.74508365e-01 7.78038545e-01 8.47179333e-01 9.41898759e-01
  8.98000460e-01 1.00399001e+00 9.48308585e-01 9.50727965e-01
  9.44539862e-01 9.54521379e-01 9.44539862e-01 1.00399001e+00
  9.50727965e-01 9.37261117e-01 1.15132693e+00 1.45405029e+00
  1.23885045e+00 1.06559667e+00 9.81397305e-01 9.83352615e-01
  9.78351208e-01 1.00000000e+00 1.21413156e+00 1.00000000e+00
  8.22625929e-01 9.91538253e-01 9.87898964e-01 9.73726514e-01
  1.01138367e+00 9.32550966e-01 1.00415197e+00 9.82564951e-01
  1.33411399e+00 9.96881917e-01 9.07833416e-01 1.00000000e+00
  1.00288719e+00 9.84791831e-01 1.00000000e+00 9.90776061e-01
  1.30224077e+00 1.00197704e+00 1.00493857e+00 9.98820836e-01
  9.97599386e-01 9.90111246e-01 1.00026804e+00 9.99640135e-01
  9.51405033e-01 1.00719641e+00 1.01318728e+00 1.06443148e+00
  7.75383243e-01 9.52164748e-01 1.00000000e+00 1.07497387e+00
  9.93559217e-01 1.00000000e+00 9.85889880e-01 1.00800864e+00
  9.89096317e-01 1.10525575e+00 1.06559667e+00 9.99043816e-01
  1.32622212e+00 1.10525575e+00 1.00025462e+00 6.68703857e-01
  8.29126910e-01 7.54026844e-01 1.21169810e+00 1.83053934e+00
  9.90596303e-01]]

As most of the variables are dummy variables, how would I know which value is it being used 1/0? Any help will be very appreciated.

$\endgroup$
2
$\begingroup$

Significance: As far as I know, sklearn does not come with a module to get p-values. You could go for statsmodels:

https://stackoverflow.com/a/55655645/9524424

Interpretation: Remember that Logit uses a logistic link function, so there is no easy on-the-go interpretation of the coefficients. You need to calculate marginal effects, so the change in $y$ when you alter some $x$, all other things equal. Stata has such module(s). However, I think sklearn has no such implementation. Python statsmodels seems to come with an option to derive marginal effects as well, but I never tried it. Look for .get_margeff.

https://www.statsmodels.org/stable/generated/statsmodels.discrete.discrete_model.DiscreteResults.get_margeff.html

$\endgroup$
  • $\begingroup$ N.B., p-values are "significance" (a measure of whether "the" coefficient is zero), not feature importances: stats.stackexchange.com/a/291239/232706 $\endgroup$ – Ben Reiniger Sep 5 '19 at 20:34
  • $\begingroup$ As for interpretation, it's not so direct, but since the linear model is for the log-odds, you can interpret the coefficients as multiplicative effects on the odds. $\endgroup$ – Ben Reiniger Sep 5 '19 at 20:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.