1
$\begingroup$

According to the documentation of Scikit-Learn Gradient Boosting Regressor:

init: estimator or ‘zero’, default=None: An estimator object that is used to compute the initial predictions. init has to provide fit and predict. If ‘zero’, the initial raw predictions are set to zero. By default a DummyEstimator is used, predicting either the average target value (for loss=’ls’), or a quantile for the other losses.

So what quantile is used for the DummyRegressor if the loss function is 'huber'? Is it the 50-quantile, ie. median?

I need this information because I am reconstructing the predictor for the Gradient Boosting Regressor for use in another software environment.

$\endgroup$

1 Answer 1

2
$\begingroup$

Yes, a GBM with Huber loss initializes with the median. The relevant bit of code is the method init_estimator of the loss class, in the file _gb_losses.py. For HuberLossFunction:

def init_estimator(self):
    return DummyRegressor(strategy='quantile', quantile=.5)

(source)

$\endgroup$
2
  • $\begingroup$ Thanks. I am somewhat confused. Is it the median of the x's to be tested? Shouldn't it be the median of the y's of the original samples? $\endgroup$ Commented Dec 26, 2020 at 22:19
  • $\begingroup$ It is indeed the median of the training y. See e.g. the documentation for DummyRegressor. $\endgroup$
    – Ben Reiniger
    Commented Dec 27, 2020 at 0:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.