I have a dataset and I have built an XGBClassifier model from it. Without hyperparameter tuning, the model performs fairly well in training but on test which have some signs of overfitting (train accuracy: 90, test accuracy: 80). After hyperparameter tuning (set the L1, L2, eta, gamma, learning_rate), the model seems to be better in terms of overfitting but the accuracy has been reduced (train accuracy: 70, test accuracy: 69). I have utilized scale_pos_weight to balance the binary target variable. Any suggestion on how could I further enhance the accuracy while not overfitting/ underfitting the model? I have tried all possible ways that I know as a beginner.
The reduction in evaluation metric values between training and test is less important than the overall ability of a model to generalize. In your case, the first model performs much better in absolute terms (80% test accuracy) and that is currently the best model to use.
Enhancing accuracy is a question of the goals of the project. Does the model need to be better to accomplish the goals of the project?
You might have to go beyond hyperparameter fitting to improve model performance. You can try adding more data and better features.