0
$\begingroup$

I would like to train my datasets in scikit-learn but export the final Gradient Boosting Regressor elsewhere so that I can make predictions directly on another platform.

I am aware that we can obtain the individual decision trees used by the regressor by accessing regressor.estimators[].tree_. What I would like to know is how to fit these decision trees together to make the final regression predictor.

$\endgroup$
2
  • $\begingroup$ You can directly save the model to disk and load it on the other platform. Why you want to do this complex exercise. Please explain a bit more in detail $\endgroup$
    – 10xAI
    Dec 23, 2020 at 17:34
  • $\begingroup$ By 'another platform' I mean a software environment other than Python. It is MQL5 used for forex trading. $\endgroup$ Dec 23, 2020 at 22:52

1 Answer 1

1
$\begingroup$

There are two estimators i.e. The initial predictor and the sub-estimators

init_estimator
The estimator that provides the initial predictions. Set via the init argument or loss.init_estimator.
estimators_
ndarray of DecisionTreeRegressor of shape (n_estimators, 1)
The collection of fitted sub-estimators.

Prediction after the first (i.e. init) estimator is controlled by the learning rate.

You can get the prediction as done in the below code -

trees = model.estimators_

x  = x_test.iloc[10,:].values # A sample X to be predicted
y_pred = model.init_.predict(x.reshape(1, -1)) # prediction from init estimator

for tree in trees:
    pred = tree[0].predict(x.reshape(1, -1)) # prediction from sub-estimator

    y_pred = y_pred + model.learning_rate*pred  # Summing with LR
y_pred
$\endgroup$
4
  • $\begingroup$ Interesting. I will test it out tomorrow. So we just use the initial sample repeatedly without having to worry about deviation from y, ie. the gradient? $\endgroup$ Dec 24, 2020 at 10:30
  • $\begingroup$ Not initial sample, it's the "x" to be predicted e.g. x_test. Trees are already fitted. $\endgroup$
    – 10xAI
    Dec 24, 2020 at 11:53
  • $\begingroup$ Great. The figures matched precisely! $\endgroup$ Dec 25, 2020 at 3:32
  • $\begingroup$ You seem to be knowledgeable in this field. I have some other questions. Could you please take a look? datascience.stackexchange.com/questions/87122/… datascience.stackexchange.com/questions/87121/… $\endgroup$ Dec 25, 2020 at 7:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.