How to save a lightGBM model that updates predictions after each fold?

Question

Hi When I use gradient boosting on Kaggle and large data sets, I run a code like this:

%%time
folds = StratifiedKFold(n_splits=5, shuffle=True, random_state=15)
oof = np.zeros(len(train))
predictions = np.zeros(len(test))
predictions_train = np.zeros(len(train))
feature_importance_df = pd.DataFrame()

start = time.time()


for fold_, (trn_idx, val_idx) in enumerate(folds.split(train.values, target.values)):
    print("fold n°{}".format(fold_))
    trn_data = lgb.Dataset(train.iloc[trn_idx][features], label=target.iloc[trn_idx])# categorical_feature=categorical_feats)
    val_data = lgb.Dataset(train.iloc[val_idx][features], label=target.iloc[val_idx])#, categorical_feature=categorical_feats)

    num_round = 10000
    clf = lgb.train(param, trn_data, num_round, valid_sets = [trn_data, val_data], verbose_eval=100, early_stopping_rounds = 200)
    oof[val_idx] = clf.predict(train.iloc[val_idx][features], num_iteration=clf.best_iteration)

    fold_importance_df = pd.DataFrame()
    fold_importance_df["feature"] = features
    fold_importance_df["importance"] = clf.feature_importance()
    fold_importance_df["fold"] = fold_ + 1
    feature_importance_df = pd.concat([feature_importance_df, fold_importance_df], axis=0)

    predictions += clf.predict(test[features], num_iteration=clf.best_iteration) / folds.n_splits
    predictions_train += clf.predict(train[features], num_iteration=clf.best_iteration) / folds.n_splits
    feature_importance_df.to_csv('feature_importance.csv',index=False)
pd.to_pickle(predictions_train,'X.pkl')
pd.to_pickle(predictions,'predictions.pkl')
print("CV score: {:<8.5f}".format(log_loss(target, oof)))

This produces good results and prevents overfitting, as I understand it the predictions are updated after each fold fitting. But is there an easy way to save this model for deployment.

As I understand it I could save the model like:

pd.to_pickle('model_fold_{}.pkl'.format(fold_),clf)
pd.to_pickle('model_best_iteration_{}.pkl'.format(fold_),clf.best_iteration)

and then load them all in, and then have a deployment script, concatenating each model on top of each other, so 5 models loaded in.

Is there a simpler way to do this?