Working on a data set similar to fraud but not monetary transactions. Here are the steps that I have taken on the modeling side:

Convert Some of the categorical into numerical (One hot encode) Over Sampled using SMOTE

  • Scaled X_train

  • PCA X_train - fit_transform

  • on the prediction side (done the same scaled X_test, PCA transform

    pca = PCA(n_components=0.95) X_train = pca.fit_transform(X_train)

I have done GridSearch to pick optimal parameters and picked the parameter when AUC is around 0.95.

My error rate is around 70%. My understanding was PCA and Scale should help tree based algorithms.

XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,
              colsample_bynode=1, colsample_bytree=0.6, eval_metric='auc',
              gamma=0.5, gpu_id=-1, importance_type='gain',
              interaction_constraints='', learning_rate=0.1, max_delta_step=0,
              max_depth=4, min_child_weight=100, missing=nan,
              monotone_constraints='()', n_estimators=100, n_jobs=16,
              num_parallel_tree=1, random_state=0, reg_alpha=0, reg_lambda=1,
              scale_pos_weight=1, subsample=1.0, tree_method='exact',
              use_label_encoder=False, validate_parameters=1, verbosity=None)
Parameters:  {'colsample_bytree': 0.6, 'gamma': 0.5, 'learning_rate': 0.1, 'max_depth': 4, 'min_child_weight': 100, 'subsample': 1.0}
Highest AUC: 0.95
  • $\begingroup$ Are you reporting test error rate, and search AUC? Is the test data left unbalanced? Did you try a different decision threshold? Did you properly scale/PC-ize the test data using transform and not fit_transform? $\endgroup$ May 12 '21 at 16:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.