I have a dataset of 3500 observations x 70 features which is my training set and I also have a dataset of 600 observations x 70 features which is the test set.
The target is to classify observations correctly either as 0 or 1. 2000 observations of the training set are 0 and the rest 1600 of them are 1.
I aim at the highest possible recall for precision>=90%.
I did grid search for ensemble algorithms only in relation to number of trees (from 50 to 650 trees). Analytically the best recall results for precision >= 90% for each of the algorithms are the following:
Random Forest (375 trees)
from sklearn.ensemble import RandomForestClassifier
classifier = RandomForestClassifier(random_state=0, n_estimators=375, class_weight='balanced')
classifier.fit(X_train, y_train)
- Precision: 90%
- Recall: 24%
Xgboost (550 trees)
from xgboost import XGBClassifier
classifier = XGBClassifier(n_estimators=n_trees, seed=0, scale_pos_weight=1.5)
classifier.fit(X_train, y_train, eval_metric='map')
- Precision: 90%
- Recall: 15%
Why Xgboost is performing so much worse than the Random Forest?