2
$\begingroup$

I ran a logistic regression (statsmodel) on my data with 60 features using the below code

import statsmodels.api as sm
logit_model=sm.Logit(y_train,X_train_std)
result=logit_model.fit()
print(result.summary())

I was able to see that few variables had negative coefficient and few had positive coefficients.

Am I right to understand that irrespective of sign of coefficients, all the below variables are significant predictors that influence the outcome?

Or does negative coefficient mean they don't have any influence on the model outcome? But p-value is significant. Am a bit confused. Can you help in simple terms please

The below output shows the records whose p-values were less than 0.05

enter image description here

Improve this question
$\endgroup$
11
  • $\begingroup$ the coefficient magnitude is what's relevant, the sign is just telling you whether the predicted probability grows when the feature grows (positive coefficient) or decreases when the feature grows (negative coefficient). $\endgroup$
    – Manu Valdés
    Dec 18, 2019 at 7:44
  • $\begingroup$ So, since I am predcting a binary outcome (disease occur or not), we need to have both positive and negative coefficient variables (with signifncant p-values). Am I right? $\endgroup$
    – The Great
    Dec 18, 2019 at 7:50
  • $\begingroup$ not necessarily, it's perfectly normal to have all positive, all negative, or both positive and negative coefficients. $\endgroup$
    – Manu Valdés
    Dec 18, 2019 at 10:26
  • $\begingroup$ Yeah, so positive coefficients indicate majorly influencing one class while negative coefficients indicate majorly influencing the other class. Right? $\endgroup$
    – The Great
    Dec 18, 2019 at 10:28
  • $\begingroup$ influencing is an awkward word choice but yes. $\endgroup$
    – Manu Valdés
    Dec 18, 2019 at 10:55

1 Answer 1

Reset to default
4
$\begingroup$

They are all significant but for certain thing.

What do I mean? You are predicting evidence, i.e. first column of the following picture:

enter image description here

In other words you have "linear regression part"+ instead of y you have evidence. So changing values of independnet variable X (positive or negative) will influence different binary class (0 or 1), hence different values are significant for different thing. (they add some info)

Improve this answer
$\endgroup$
5
  • $\begingroup$ So usually positive coefficients influence label 1 and negative coefficients influence Lable 0. My label 0 indicates - No disease and Label 1 indicates - Disease present $\endgroup$
    – The Great
    Dec 18, 2019 at 9:14
  • $\begingroup$ Or the other way around, it depends how you defined it. But you can check it easily: Take some columns of X that you can reason about (i.e. somehing known) and look if these values are positive and log probability is xyz than you can deduce what influences what. ALTERNATIVE. Use feature importance and see exactly what influences what class $\endgroup$
    – Noah Weber
    Dec 18, 2019 at 9:17
  • $\begingroup$ By defining what do you mean? I have defined people who have disease as 1 and don't have disease as 0 $\endgroup$
    – The Great
    Dec 18, 2019 at 9:18
  • $\begingroup$ I mean if rows of X map to labels 1 or 0. If you switch the rows that point to label 1 now to point (be instances ) of label 0 you will get opposite interpretation. $\endgroup$
    – Noah Weber
    Dec 18, 2019 at 9:20
  • $\begingroup$ Sure. dont forget to accept if you are satisfied. $\endgroup$
    – Noah Weber
    Dec 18, 2019 at 9:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.