# Quantile Regression is inconsistent (lower quantiles predicting higher values at times)

While using scikit-learn's GradientBoostingRegressor's "quantile" loss, I noticed that when I try different values of q to fit the data at 0.05 (5%) increments, there are instances when predicting a value where a higher quantile will output a lower prediction of the value. This shouldn't be the case, if I'm not mistaken.

Here is the code:

from sklearn.ensemble import GradientBoostingRegressor
resultsdf = pd.DataFrame(test_y)
clf = GradientBoostingRegressor(loss='quantile', n_estimators=1000, max_depth=1, learning_rate=0.5, min_samples_leaf=9, min_samples_split=9, random_state=103)

for alpha in np.linspace(0.05, 0.95, num=19):
clf.set_params(alpha=alpha)
clf.fit(train_x, train_y)
y = clf.predict(test_x)
resultsdf['{:.2f}'.format(alpha)] = y


The resultsdf dataframe shows that sometimes, lower quantiles predict higher values. train_x has 3 features, and the training size of it is ~200 samples.

Any ideas why this is the case?