ML Project - Achieve 2 Objectives

Question

I have a dataset with 5K records focused on binary classification. I am posting it here to seek your suggestions on project methodology

Currently what is my objective is

1) Run statsmodel logistic regression to find risk factors that influence the outcome

2) Then build a predictive model based on best features (may or may not include risk factors). because as you may know not all significant variables are good predictors.

Though I can use scikit-learn logistic regression to build a predictive model but I am planning to go with Xgboost because it provides better performance in my dataset (non-linear data slightly imbalanced)

I do step one because I have to find what are the risk factors that influence the outcome, so I am doing it. (ex: risk factors that influence customer to default in loan repayment) You know where we get p-values and find significant risk factors.

In 2nd step, I build predictive model because I realized through running the built model that not all risk factors are good predictors. So in the end, I include new set of features that help in better prediction along with risk factors

Do you think that I am right in having/approaching this as two objectives problem?

Do you think what I am doing is redundant or am proceeding in the right direction?

Do you think there is no reason to use 2 algorithms separately?

Do you have any suggestions or tips to make it easy to achieve my objective?

Answer 1

Xgboost does the feature selection for you. If you want to report how valuable certain features are for prediction, print the feature importances. Those will, however, just tell you "feature $x_1$is very important for predicting the outcome, feature $x_2$ is nearly useless for predicting the outcome, etc.". To get risk factors with p-values, you need a more interpretable model. You have to use linear classification, for example. Then you can make statements like "high $x_1$ correlates with a positive outcome".

If you really want to only use a subset of features, train an Xgboost model on a validation dataset and drop features that have low importance. Then run an Xgboost model with the remaining features on the remaining training set.

You need to think about what you want: Do you want to explain observations and extract explicit knowledge from data and try to find possible causal links? Use linear methods. Do you want to predict an outcome for new patient data? Use gradient boosting. You can obviously do both.