# Trying to beat random forest with xgboost

I have a small time series dataset of about 3000 samples and 5 features. With xgboost, my predictions seem biased (consistently overestimating the target). No matter how many estimators I throw at the problem along with hyperparameter tuning, I can't seem to beat a random forest. How can I go about diagnosing this?

rf = RandomForestRegressor(n_estimators=1000,
max_features=1,
min_samples_leaf=20,
random_state=0, n_jobs=-1)

xgb = xgb.XGBRegressor(n_estimators=100000,
max_depth=1,
learning_rate=.0001,
min_child_weight=1,
subsample=.1,
colsample_bytree=.1,
base_score=0)


• Isn’t it possible random forest is just the best algorithm for your data? – astel Sep 22 '19 at 3:47
• There's probably nothing to diagnose, your mistake was to assume that decision trees are too old for competing with the young generation ;) – Erwan Sep 23 '19 at 1:25
• I just thought any problem RF can do, XGB can do better! Kidding aside, it's frustrating to not know why certain algos fit certain data better. – kliao Sep 23 '19 at 21:37
• en.wikipedia.org/wiki/No_free_lunch_theorem – Erwan Sep 24 '19 at 12:05