5
$\begingroup$

I want to obtain the prediction intervals of my xgboost model which I am using to solve a regression problem. I am using the python code shared on this blog, and not really understanding how the quantile parameters affect the model (I am using the suggested parameter values on the blog). When I apply this code to my data, I obtain nonsense results, such as negative predictions for my target values while my target values are always over 10K. I don't understand how should this code vary according to my data and would really appreciate any help.

Differences in my data to the data that is used on the blog are:

  • My distribution is Poisson like.
  • I have over 100 features.

Note: I tried tuning the delta, threshold and var parameters, but they don't seem to have a controllable effect on the results and predictions remains nonsense.

$\endgroup$
1
$\begingroup$

To produce confidence intervals for xgboost model you should train several models (you can use bagging for this). Each model will produce a response for test sample - all responses will form a distribution from which you can easily compute confidence intervals using basic statistics. You should produce response distribution for each test sample.

this answer is provided here: https://stackoverflow.com/questions/37418938/how-to-obtain-a-confidence-interval-or-a-measure-of-prediction-dispersion-when-u

$\endgroup$
1
  • 2
    $\begingroup$ The OP is about prediction intervals, not c.i. $\endgroup$
    – Michael M
    Jun 10 '18 at 7:54
0
$\begingroup$

Try the following code. It must work fine. It might take a lot of time (more than 100 features).

Change max_depth to 6 if you want more accuracy. (because of 100 features.)

We can change learning_rate between 0 and 1, to improve the efficiency.

import xgboost as xgb
model_xgb = xgb.XGBRegressor(colsample_bytree=0.4603, gamma=0.0468, 
                             learning_rate=0.05, max_depth=3, 
                             min_child_weight=1.7817, n_estimators=4200,
                             reg_alpha=0.4640, reg_lambda=0.8571,
                             subsample=0.5213, silent=1,
                             nthread=-1)

X_train, X_test, Y_train, Y_test= train_test_split(X, Y, random_state= 0)
def model_score_error(model):
    prepared_model=model.fit(X_train, Y_train)
    x=prepared_model.score(X_test,Y_test)
    print('Score: ',x)
    Target_predicted=prepared_model.predict(X_test) 
    MSE=mean_squared_error(Y_test,Target_predicted) 
    print('mean square error', MSE)

model_score_error(model_xgb)
$\endgroup$
3

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.