I want to train by Xgboost algorithm and predict directly using the trees while testing. Precisely, speaking I don't want to keep the model weights in any file like "joblib" and load it while prediction. I wish to keep the decision trees in a "transparent file" like csv. The code to train and get the decision trees can be as follows,

import xgboost as xgb
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
import numpy as np
import tempfile

# Load the California Housing dataset
california = fetch_california_housing()
X, y = california.data, california.target

# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Convert the training dataset into DMatrix format
dtrain = xgb.DMatrix(X_train, label=y_train)

# Set the parameters for XGBoost training
params = {
    'objective': 'reg:squarederror',  # Regression objective
    'max_depth': 3,  # Maximum depth of a tree
    'eta': 0.1,  # Learning rate
    'seed': 42  # Random seed for reproducibility

# Train the XGBoost model with 10 trees
model = xgb.train(params=params, dtrain=dtrain, num_boost_round=10)

# Retrieve the list of trees
trees = model.get_dump()

The trees object is a list of 10 trees. Each trees are in a string datatype. A particular tree looks like

0:[f0<5.0864501] yes=1,no=2,missing=1
    1:[f0<3.07429981] yes=3,no=4,missing=3
        3:[f2<4.31418896] yes=7,no=8,missing=7
        4:[f5<2.41788673] yes=9,no=10,missing=9
    2:[f0<6.88754988] yes=5,no=6,missing=5
        5:[f5<2.67065763] yes=11,no=12,missing=11
        6:[f0<7.81515026] yes=13,no=14,missing=13

Now I want to make prediction using those trees, I couldn't find any built-in function to do it. I may make my own function (by the formula $$ y^* = y_0 + \Sigma_{i = 1}^{i = N}\eta * f_i $$ where $y^*$ is the predicted function, $y_0$ is the average value of the target or the base learner, $\eta$ is the learning rate and $f_i$ is the decision tree at $i$-th step) for doing this job, but still looking for any built in function as it would be optimized and less tedious for me. Any advise regarding this issue will be highly appreciated.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.