1
$\begingroup$

According to the documentation of Scikit-Learn Gradient Boosting Regressor:

init: estimator or ‘zero’, default=None: An estimator object that is used to compute the initial predictions. init has to provide fit and predict. If ‘zero’, the initial raw predictions are set to zero. By default a DummyEstimator is used, predicting either the average target value (for loss=’ls’), or a quantile for the other losses.

So what quantile is used for the DummyRegressor if the loss function is 'huber'? Is it the 50-quantile, ie. median?

I need this information because I am reconstructing the predictor for the Gradient Boosting Regressor for use in another software environment.

$\endgroup$
2
$\begingroup$

Yes, a GBM with Huber loss initializes with the median. The relevant bit of code is the method init_estimator of the loss class, in the file _gb_losses.py. For HuberLossFunction:

def init_estimator(self):
    return DummyRegressor(strategy='quantile', quantile=.5)

(source)

$\endgroup$
2
  • $\begingroup$ Thanks. I am somewhat confused. Is it the median of the x's to be tested? Shouldn't it be the median of the y's of the original samples? $\endgroup$ – Chong Lip Phang Dec 26 '20 at 22:19
  • $\begingroup$ It is indeed the median of the training y. See e.g. the documentation for DummyRegressor. $\endgroup$ – Ben Reiniger Dec 27 '20 at 0:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.