I am working on a classification problem. I have two models:
Logistic regression model
Random Forest model
For the first model, if I choose the only predictors with p-values<0.05 I will reduce the accuracy, recall, etc. The IV (and WOE) predictors selection are not the same as the predictors that comply with the p-value rule.
On the other hand, Random Forest does not give p-values, only importance metrics, which are similar the IV predictors.
So, which methods should use to select the right predictors so I can compare the two models?
Thanks for reading this and I hope you can help me. I will be glad to give more details.