I have a lot of features and many are correlated, so I performed dimensionality reduction. I then used these components in binary classification and got high accuracy. I also performed feature importance to see which of the features contributed the best to the separation of the 2 groups and found out that PC5 was the best to separate the groups. I looked at the feature loadings inside PC5 and saw that there are 2 features that loaded significantly higher than any other features on this component. Is it possible to say that those features contribute to the separation of the groups i was trying to classify more than other features? or would this not be the case?

  • $\begingroup$ I'd say yes, the logic seems good to me. What you could do to prove this is to train a simple independent model (say decision tree for instance) with only these original 2 features, and see how it performs. $\endgroup$
    – Erwan
    Jan 13, 2023 at 16:25
  • $\begingroup$ Thank you very much for confirming that, Erwan. I also did what you recommended (the simple independent model training), and it showed high performance as well. PS: I'm new here so i don't have the privilege to up vote your comment, but i do appreciate it. $\endgroup$
    – Din
    Jan 13, 2023 at 16:46


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.