0
$\begingroup$

I am working on highly imbalanced dataset and trying to increase accuracy(metric: roc_auc) of my model which is hovering around 82-83%. This is part of an internal ML competition and people who are at the top have accuracy around 85-88%.

I am just wondering what else I can do to improve my model's accuracy. Any suggestion or tips would be appreciated.

Details of training dataset :

The training data shape is : (166573, 14)

Distribution of features :

As you can see, only the first 4 columns go to different max values. Rest of the columns have either 1 or 0 value (max: 1, min: 0)

enter image description here

Scaling features :

X['scaled_distance']= sc.fit_transform(X['distance'].values.reshape(-1,1))
X['scaled_visit_count'] = sc.fit_transform(X['visit_count'].values.reshape(-1,1))
X['scaled_tier'] = sc.fit_transform(X['tier'].values.reshape(-1,1))

Null Handling :

train['tier'].fillna(round(train['tier'].mean(),2),inplace=True)

At last, I have tried different models (Xgboost, Random Forest with SMOTE, lightbgm etc..) I have got best results with lightbgm with some tuned parameters..

lgbm.fit(X_train,y_train)
LGBMClassifier(bagging_fraction=0.8, bagging_freq=15, boosting_type='gbdt',
        class_weight=None, colsample_bytree=1.0, feature_fraction=0.5,
        importance_type='split', is_unbalance=True, learning_rate=0.01,
        max_depth=7, min_child_samples=20, min_child_weight=0.001,
        min_split_gain=0.0, n_estimators=520, n_jobs=-1, num_leaves=40,
        objective=None, random_state=10, reg_alpha=0.0, reg_lambda=0.0,
        silent=True, subsample=1.0, subsample_for_bin=200000,
        subsample_freq=0)

Please refer full code here : https://github.com/PraveenKS30/ML/blob/master/Surge2019/Surge%20Pre%20Machine%20Learning.ipynb

I am not sure what else I can do to improve my accuracy.. Should I preprocess in different way ? Should I try neural network now? Please suggest.

$\endgroup$
  • $\begingroup$ Just out of curiosity, why do you want to improve accuracy for a highly imbalanced dataset? Is that a competition request? $\endgroup$ – sentence Apr 26 at 18:55
  • $\begingroup$ Yup, it's for the rank on leaderboard.. even if I manage to improve model accuracy by 1%.. I will be in top 10.. $\endgroup$ – Praveenks Apr 26 at 19:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.