I would like to train my datasets in scikit-learn but export the final Gradient Boosting Regressor elsewhere so that I can make predictions directly on another platform.

I am aware that we can obtain the individual decision trees used by the regressor by accessing regressor.estimators[].tree_. What I would like to know is how to fit these decision trees together to make the final regression predictor.

  • $\begingroup$ You can directly save the model to disk and load it on the other platform. Why you want to do this complex exercise. Please explain a bit more in detail $\endgroup$
    – 10xAI
    Dec 23, 2020 at 17:34
  • $\begingroup$ By 'another platform' I mean a software environment other than Python. It is MQL5 used for forex trading. $\endgroup$ Dec 23, 2020 at 22:52

1 Answer 1


There are two estimators i.e. The initial predictor and the sub-estimators

The estimator that provides the initial predictions. Set via the init argument or loss.init_estimator.
ndarray of DecisionTreeRegressor of shape (n_estimators, 1)
The collection of fitted sub-estimators.

Prediction after the first (i.e. init) estimator is controlled by the learning rate.

You can get the prediction as done in the below code -

trees = model.estimators_

x  = x_test.iloc[10,:].values # A sample X to be predicted
y_pred = model.init_.predict(x.reshape(1, -1)) # prediction from init estimator

for tree in trees:
    pred = tree[0].predict(x.reshape(1, -1)) # prediction from sub-estimator

    y_pred = y_pred + model.learning_rate*pred  # Summing with LR
  • $\begingroup$ Interesting. I will test it out tomorrow. So we just use the initial sample repeatedly without having to worry about deviation from y, ie. the gradient? $\endgroup$ Dec 24, 2020 at 10:30
  • $\begingroup$ Not initial sample, it's the "x" to be predicted e.g. x_test. Trees are already fitted. $\endgroup$
    – 10xAI
    Dec 24, 2020 at 11:53
  • $\begingroup$ Great. The figures matched precisely! $\endgroup$ Dec 25, 2020 at 3:32
  • $\begingroup$ You seem to be knowledgeable in this field. I have some other questions. Could you please take a look? datascience.stackexchange.com/questions/87122/… datascience.stackexchange.com/questions/87121/… $\endgroup$ Dec 25, 2020 at 7:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.