4
$\begingroup$

I am working on a regression problem, where I want to modify the loss function in xgboost library such that my predictions should never be lesser than the actual value. I have written this code:

def custom_loss(preds, dtrain):
    labels = dtrain.get_label()
    df = preds - labels
    df = pd.DataFrame(df, columns=['val'])
    df['valg'] = df['val'].apply(lambda x: 10*abs(x) if x<0 else x)
    grad = df['valg'].as_matrix()
    return preds-labels, grad

This essentially means that I want to penalize those predictions which are lesser than my actual value more. However, this isn't working and there is no improvement in my predictions. Can anyone help me figure out where I am going wrong? Thanks.

Whole Python script:

params = {"booster" : "gbtree",
  "eta": 0.20,
  "max_depth": 4,
  "subsample": 0.75,
  "colsample_bytree": 0.65,
  "silent": 1,
  "eval_metric": "rmse",
  }
num_round = 400

def custom_loss(preds, dtrain):
    labels = dtrain.get_label()
    df = preds - labels
    df = pd.DataFrame(df, columns=['val'])
    df['valg'] = df['val'].apply(lambda x: 5*abs(x) if x<0 else x)
    grad = df['valg'].as_matrix()
    return preds-labels, grad


dtrain = xgb.DMatrix(X_train.drop('price_act', axis=1), 
label=X_train['price_act'])
dtest = xgb.DMatrix(X_test.drop('price_act',axis=1), 
label=X_test['price_act'])

watchlist = [(dtrain,'train'), (dtest,'eval')]

bst = xgb.train(params, dtrain, num_round, watchlist, custom_loss)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.