Hi When I use gradient boosting on Kaggle and large data sets, I run a code like this:

folds = StratifiedKFold(n_splits=5, shuffle=True, random_state=15)
oof = np.zeros(len(train))
predictions = np.zeros(len(test))
predictions_train = np.zeros(len(train))
feature_importance_df = pd.DataFrame()

start = time.time()

for fold_, (trn_idx, val_idx) in enumerate(folds.split(train.values, target.values)):
    print("fold n°{}".format(fold_))
    trn_data = lgb.Dataset(train.iloc[trn_idx][features], label=target.iloc[trn_idx])# categorical_feature=categorical_feats)
    val_data = lgb.Dataset(train.iloc[val_idx][features], label=target.iloc[val_idx])#, categorical_feature=categorical_feats)

    num_round = 10000
    clf = lgb.train(param, trn_data, num_round, valid_sets = [trn_data, val_data], verbose_eval=100, early_stopping_rounds = 200)
    oof[val_idx] = clf.predict(train.iloc[val_idx][features], num_iteration=clf.best_iteration)

    fold_importance_df = pd.DataFrame()
    fold_importance_df["feature"] = features
    fold_importance_df["importance"] = clf.feature_importance()
    fold_importance_df["fold"] = fold_ + 1
    feature_importance_df = pd.concat([feature_importance_df, fold_importance_df], axis=0)

    predictions += clf.predict(test[features], num_iteration=clf.best_iteration) / folds.n_splits
    predictions_train += clf.predict(train[features], num_iteration=clf.best_iteration) / folds.n_splits
print("CV score: {:<8.5f}".format(log_loss(target, oof)))

This produces good results and prevents overfitting, as I understand it the predictions are updated after each fold fitting. But is there an easy way to save this model for deployment.

As I understand it I could save the model like:


and then load them all in, and then have a deployment script, concatenating each model on top of each other, so 5 models loaded in.

Is there a simpler way to do this?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.