I have following dataset.

  • Total 31 columns including Target.
  • Target column has value of either 1 or 0.
  • This is balanced dataset.
  • All 30 Feature columns also have value of either 1 or 0.
  • All these 30 Features columns have very less importance on Target column.

Now I have tried different classification boosting algorithm like XGBRFClassifier, AdaBoostClassifier, CatBoostClassifier, LGBMClassifier, GradientBoostingClassifier.

But I am getting prediction result very low (below 85 percent). Actually getting overfitting problem. Accuracy in training is around 99 % while in validation it's 85%. I have also tried with Deep Learning (ANN), but not getting proper result.

Please suggest me how to solve this overfitting problem.

  • $\begingroup$ Welcome to DataScienceSE. 85% accuracy is not necessarily low, depends on the problem. Did you try simple models like Naive Bayes, decision trees? They might fix the overfitting. $\endgroup$
    – Erwan
    Jul 28 at 17:29


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy