I am working on highly imbalanced dataset and trying to increase accuracy(metric:
roc_auc) of my model which is hovering around 82-83%. This is part of an internal ML competition and people who are at the top have accuracy around 85-88%.
I am just wondering what else I can do to improve my model's accuracy. Any suggestion or tips would be appreciated.
Details of training dataset :
The training data shape is : (166573, 14)
Distribution of features :
As you can see, only the first 4 columns go to different max values. Rest of the columns have either 1 or 0 value (max: 1, min: 0)
Scaling features :
X['scaled_distance']= sc.fit_transform(X['distance'].values.reshape(-1,1)) X['scaled_visit_count'] = sc.fit_transform(X['visit_count'].values.reshape(-1,1)) X['scaled_tier'] = sc.fit_transform(X['tier'].values.reshape(-1,1))
Null Handling :
At last, I have tried different models (
Random Forest with SMOTE,
lightbgm etc..) I have got best results with lightbgm with some tuned parameters..
lgbm.fit(X_train,y_train) LGBMClassifier(bagging_fraction=0.8, bagging_freq=15, boosting_type='gbdt', class_weight=None, colsample_bytree=1.0, feature_fraction=0.5, importance_type='split', is_unbalance=True, learning_rate=0.01, max_depth=7, min_child_samples=20, min_child_weight=0.001, min_split_gain=0.0, n_estimators=520, n_jobs=-1, num_leaves=40, objective=None, random_state=10, reg_alpha=0.0, reg_lambda=0.0, silent=True, subsample=1.0, subsample_for_bin=200000, subsample_freq=0)
Please refer full code here : https://github.com/PraveenKS30/ML/blob/master/Surge2019/Surge%20Pre%20Machine%20Learning.ipynb
I am not sure what else I can do to improve my accuracy.. Should I preprocess in different way ? Should I try neural network now? Please suggest.