Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I am doing feature selection using Sklearn:
Target variable is binary classes. The training set is standardized.
How should I interpret when a certain feature shows significant importance in Random Forest estimator, but negative coefficient in Logistic Regression?
Negative coefficient in logistic regression means negative relationship between predictor variable and the response variable.
Example- in a model price can be a predictor and will have negative relationship with binary response variable product purchased or not.
And neagtive coefficient in logistic regression does not mean relationship has low strength, it only means changes in predictor has a reverse effect on response variable and if coefficient is highly negative it means feature is very important and small changes in it impacts response but in reverse direction.
Feature importance does not tell you nature or direction of relationship but only tells you strength of relationship so they are never negative. Coefficients in logistic regression tells you both strength and direction or nature (positive or negative) of relationship. Also high importance in random forest means strong relationship between predictor and response but importance column derived from tree based models is silent on nature or direction.
Hope this helps.
