0
$\begingroup$

While using scikit-learn's GradientBoostingRegressor's "quantile" loss, I noticed that when I try different values of q to fit the data at 0.05 (5%) increments, there are instances when predicting a value where a higher quantile will output a lower prediction of the value. This shouldn't be the case, if I'm not mistaken.

Here is the code:

from sklearn.ensemble import GradientBoostingRegressor
resultsdf = pd.DataFrame(test_y)
clf = GradientBoostingRegressor(loss='quantile', n_estimators=1000, max_depth=1, learning_rate=0.5, min_samples_leaf=9, min_samples_split=9, random_state=103)

for alpha in np.linspace(0.05, 0.95, num=19):
    clf.set_params(alpha=alpha)
    clf.fit(train_x, train_y)
    y = clf.predict(test_x)
    resultsdf['{:.2f}'.format(alpha)] = y

The resultsdf dataframe shows that sometimes, lower quantiles predict higher values. train_x has 3 features, and the training size of it is ~200 samples.

Any ideas why this is the case?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.