XGBoostRegressor is not performing better than AdaBoostRegressor for the same set of parameters for some reason. Since my dataset is big, I made an example using sklearn's make_regression as follows.

from sklearn.ensemble import AdaBoostRegressor
from sklearn.datasets import make_regression
from sklearn.tree import DecisionTreeRegressor
from sklearn.linear_model import LinearRegression
from xgboost import XGBRegressor

X, y = make_regression(n_samples=10000, n_features=1, n_informative=1,
                       random_state=0, noise=1,shuffle=False)

regr = LinearRegression()
regr.fit(X, y)
print regr.score(X, y)

regr = AdaBoostRegressor(DecisionTreeRegressor(max_depth=6),n_estimators=10,random_state=0)
regr.fit(X, y)
print regr.score(X, y)

regr = XGBRegressor(max_depth=6,n_estimators=10,random_state=0)
regr.fit(X, y)
print regr.score(X, y)



I want the same max_depth and n_estimators for comparison purposes. What, if any, is the hyper-parameter making the difference?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.