Checkpoints in Sklearn

Question

Is there a way to save the current state of your experiment so that you can pick up from where you left off in Sklearn similar to checkpoints in Keras?

Answer 1

I think the closest you can get is with either the warm_start parameter or the partial_fit call. They are available in some models and allow you to train a model several times without losing progress.

From sklearn docs:

warm_start : bool, optional, default False

When set to True, reuse the solution of the previous call to fit as initialization, otherwise, just erase the previous solution.

and:

partial_fit(X, y, classes=None, sample_weight=None)[source]

Perform one epoch of stochastic gradient descent on given samples.

Internally, this method uses max_iter = 1. Therefore, it is not guaranteed that a minimum of the cost function is reached after calling it once. Matters such as objective convergence and early stopping should be handled by the user.

Which one to use depends on what model you are using and when you want to make your checkpoints. But if you for example use a RandomForestClassifier which has warm_start you could do the following:

# set warm_start so model to avoid erasing model between fits
clf = RandomForestClassifier(warm_start=True)
number_of_checkpoints = 10

for checkpoint in range(number_of_checkpoints):

    # Load only a subset of the data and train on it
    X, y = load_data_batch(batches=number_of_checkpoints, current_batch=checkpoint)
    clf.fit(X, y)

    # Save model checkpoint for each fit
    with open('path/to/models/random_forest_ckp_{}.p'.format(checkpoint), 'wb') as f:
        pickle.dump(clf, f)