I have trained two boosted classifiers on same data with same features, but I am getting two different set of feature importance from the models.

Few high ranked features (suppose top 10) from one boosted classifier is almost 5-6 ranks below when compared with the top 10 of other boosted classifier.

Is there anything wrong with this, or its totally okay to have this kind of behavior as it depends on how a model learns.

Thanks in advance.

  • $\begingroup$ Are these continuous or categorical? Are they highly co-linear? Do the models perform similarly? How many features? Sounds like maybe the features contain similar information and are just being selected differently. $\endgroup$ – Hobbes Jul 20 '17 at 14:27
  • $\begingroup$ Have you tried using a different classifier as well? It would be interesting to see if there's a consensus about feature importance. Try using a random forest after you ensure your features are independent, and seeing which features the random forest thinks are most important. Then compare that ranked list with the one from your boosted classifer and see if any of them agree. $\endgroup$ – StatsSorceress Jul 20 '17 at 16:20
  • $\begingroup$ @Hobbes They are all continous. I need to check that whether they are highly collinear or not. Yes , the models perform similarly as they are giving almost the same result. May be the features convey the same information but not sure about that will check that . $\endgroup$ – Anurag Upadhyaya Jul 20 '17 at 17:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.