I want to do a multi class classification with 6 classes. Whole dataset has 12750 and 56 features samples, so every class has 2125 samples. Before prediction I reduces amount of outliers by winsorization (for 1 and 99 percentile) and I reduced skewness in features which has more than 1 and less than -1 skewness by Yeo-Johnson transformation and I got dataset:
Later, of course, I splitted dataset for 80% of training data and 20% of test data and I standardised training data. I tried to use random forest, xgboost and decision tree classifiers, but I have almost 100% accuracy on training set and 20-21% accuracy on test set. Methods like increasing n_estimators doesn't help.
So, my questions are:
How can I reduce this overfitting? Is it a problem with dataset (Should I reduce number of features something like that?) or with classificators (Are they too weak for this problem?)
Is the dataset too small for this problem (Should I add more samples by method like SMOTE?)? Do classes have too less samples to good work?
Is it possible to get at least 60% accuracy after tuning hyperparameters (e.g. by method like GridSearchCV)?
P.S. I will add that correlations with target value are very poor (max +- 6%) and I see that feature importances from random forest have values from 0.0 to 0.03. I don't know if this is a normal situation.
P.S.2 I tried to change n_estimators parameters (values from 5 to 1500) and max_depth (from 1 to 100) and I can see very poor change in test accuracy (+-3%)