I am building a binary classification model where classes are imbalanced but used SMOTE, I used 4 different models to compare performance and decide which to choose. They have same train and test accuracy.When I tested the models xgboost and logistic regression only on people with credit cards,lr predicted 81% but xgboost and random forest 50% predicted right.
Model:(Train Accuracy,Test Accuracy)
- Random Forest:(0.963,0.961)
- Decision Tree:(0.962,0.961)
- Logistic Regression:(0.815, 0.815)
- Decision Tree:(0.949,0.993)
- Logistic Regression:(0.863,0.981)
- Random Forest:(0.968,0.995)