While using scikit-learn's GradientBoostingRegressor's "quantile" loss, I noticed that when I try different values of q to fit the data at 0.05 (5%) increments, there are instances when predicting a value where a higher quantile will output a lower prediction of the value. This shouldn't be the case, if I'm not mistaken.

Here is the code:

from sklearn.ensemble import GradientBoostingRegressor
resultsdf = pd.DataFrame(test_y)
clf = GradientBoostingRegressor(loss='quantile', n_estimators=1000, max_depth=1, learning_rate=0.5, min_samples_leaf=9, min_samples_split=9, random_state=103)

for alpha in np.linspace(0.05, 0.95, num=19):
    clf.fit(train_x, train_y)
    y = clf.predict(test_x)
    resultsdf['{:.2f}'.format(alpha)] = y

The resultsdf dataframe shows that sometimes, lower quantiles predict higher values. train_x has 3 features, and the training size of it is ~200 samples.

Any ideas why this is the case?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.