2
$\begingroup$

I ran a logistic regression (statsmodel) on my data with 60 features using the below code

import statsmodels.api as sm
logit_model=sm.Logit(y_train,X_train_std)
result=logit_model.fit()
print(result.summary())

I was able to see that few variables had negative coefficient and few had positive coefficients.

Am I right to understand that irrespective of sign of coefficients, all the below variables are significant predictors that influence the outcome?

Or does negative coefficient mean they don't have any influence on the model outcome? But p-value is significant. Am a bit confused. Can you help in simple terms please

The below output shows the records whose p-values were less than 0.05

enter image description here

$\endgroup$
  • $\begingroup$ the coefficient magnitude is what's relevant, the sign is just telling you whether the predicted probability grows when the feature grows (positive coefficient) or decreases when the feature grows (negative coefficient). $\endgroup$ – Manu Valdés Dec 18 '19 at 7:44
  • $\begingroup$ So, since I am predcting a binary outcome (disease occur or not), we need to have both positive and negative coefficient variables (with signifncant p-values). Am I right? $\endgroup$ – The Great Dec 18 '19 at 7:50
  • $\begingroup$ not necessarily, it's perfectly normal to have all positive, all negative, or both positive and negative coefficients. $\endgroup$ – Manu Valdés Dec 18 '19 at 10:26
  • $\begingroup$ Yeah, so positive coefficients indicate majorly influencing one class while negative coefficients indicate majorly influencing the other class. Right? $\endgroup$ – The Great Dec 18 '19 at 10:28
  • $\begingroup$ influencing is an awkward word choice but yes. $\endgroup$ – Manu Valdés Dec 18 '19 at 10:55
3
$\begingroup$

They are all significant but for certain thing.

What do I mean? You are predicting evidence, i.e. first column of the following picture:

enter image description here

In other words you have "linear regression part"+ instead of y you have evidence. So changing values of independnet variable X (positive or negative) will influence different binary class (0 or 1), hence different values are significant for different thing. (they add some info)

| improve this answer | |
$\endgroup$
  • $\begingroup$ So usually positive coefficients influence label 1 and negative coefficients influence Lable 0. My label 0 indicates - No disease and Label 1 indicates - Disease present $\endgroup$ – The Great Dec 18 '19 at 9:14
  • $\begingroup$ Or the other way around, it depends how you defined it. But you can check it easily: Take some columns of X that you can reason about (i.e. somehing known) and look if these values are positive and log probability is xyz than you can deduce what influences what. ALTERNATIVE. Use feature importance and see exactly what influences what class $\endgroup$ – Noah Weber Dec 18 '19 at 9:17
  • $\begingroup$ By defining what do you mean? I have defined people who have disease as 1 and don't have disease as 0 $\endgroup$ – The Great Dec 18 '19 at 9:18
  • $\begingroup$ I mean if rows of X map to labels 1 or 0. If you switch the rows that point to label 1 now to point (be instances ) of label 0 you will get opposite interpretation. $\endgroup$ – Noah Weber Dec 18 '19 at 9:20
  • $\begingroup$ Sure. dont forget to accept if you are satisfied. $\endgroup$ – Noah Weber Dec 18 '19 at 9:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.