I am trying to build an Uplift model using observational data. The data is consists of collections calls to customers and my objective is to predict the incremental probability due to the treatment (collection calls). I am using XGBoost algorithm for the same, and as I am using observational data, I need to do an Inverse Probability Weighting (IPW) to remove the bias caused due to the unavailability of Randomized Test and Control groups. Hence, I have predicted the probability of a customer being contacted using a logistic regression and used the probabilities to calculate IPW. Now, to remove the bias, I need to weigh each instance of my data with the IPW while building the final uplift model. Here's where I am facing the issue, when I pass these weights while training, post training xgboost predicts everything as either null or .5 for every data point in both test and training data. When I don't use these weights, the model runs fine and gives out proper probabilities. Below is the code I am using

X_train, X_test, Y_train, Y_test = model_selection.train_test_split(X, Y, train_size = 0.8,random_state = 42,stratify  = Y)
train_weights = X_train.sipw
test_weights = X_test.sipw
xgbmat_train = xgb.DMatrix(X_train,Y_train,missing=np.nan,weight=train_weights.values)
xgbmat_test = xgb.DMatrix(X_test,Y_test,missing=np.nan,weight=test_weights.values)
initial_params = {'learning_rate':0.1,'n_estimators':1000,'objective':'binary:logistic','booster':'gbtree','reg_alpha':0,
xgb_1 = xgb.train(initial_params,xgbmat_train)
prob = xgb_1.predict(xgbmat_test)

I am going crazy thinking about why this is happening, as I had previously built a model (for a different country) using the same code as above (passing IPW as weights) and it gave me perfect results. While for this particular dataset, I am unable figure out why the model is returning such results. Any help will be deeply appreciated.

  • $\begingroup$ Make sure there are no null values in the sample weights. I had the same issue, figured out it was because of nulls. Works fine when I impute the null values! $\endgroup$ – Deepak Feb 20 at 0:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.