Prohibitive size of random forest when saved to disk

Question

When saved to disk using cPickle: https://stackoverflow.com/questions/20662023/save-python-random-forest-model-to-file, my random forest is 6.57 GB.

with open('rforest.cpickle', 'wb') as f:
    cPickle.dump(rforest, f)

I want to use the forest itself to make predictions via a python API hosted on Heroku -- of course, that file size is unacceptable.

Why is the file size so large? There are 500 trees in the ensemble -- all I want to save are the completed trees themselves, since they will be used as prediction. Is it the actual nodes and edges that compose each of the 500 trees that requires nearly 7 GB of space on disk?

I used scikitlearn's randomforestregressor:

def buildForest(self, X_train, y_train):
    rf = RandomForestRegressor(n_estimators=500, verbose=1)
    rf.fit_transform(X_train, y_train)
    return rf

Also, if there is a better way to make my model accessible via API, that would also be good to know.

Update: I reduced it to 100 trees without losing much predictive power, and so now the saved size is 1.3 GB -- much more manageable, but still not great.

Answer 1

The size of each tree depends very much on its depth. Thus, change the maximal depth (max_depth). Try to set it to finite number (as opposed to the default "None") and then try to reduce this number. In addition (or as alternative) try to increase min_samples_split or min_samples_split.

You can also analyze you features and keep only important ones. The simplest way would be to have a look at the clf.feature_importances_ of your forest. (In general, finding important features is an art and science on itself.) Exclude non-relevant features and rebuild the forest.

Answer 2

Try this：

import pickle
with open('rforest.pickle', 'wb') as f:
    pickle.dump(rforest, f, -1)

Notice: with the parameter "-1" the model file size will largely be reduced.

According to the documentation:

pickle.dump(obj, file[, protocol])

Protocol version 0 is the original ASCII protocol and is backwards compatible with earlier versions of Python. Protocol version 1 is the old binary format which is also compatible with earlier versions of Python. Protocol version 2 was introduced in Python 2.3. It provides much more efficient pickling of new-style classes.

If the protocol parameter is omitted, protocol 0 is used. If protocol is specified as a negative value or HIGHEST_PROTOCOL, the highest protocol version will be used.

Answer 3

I ran into a similar issue. Even with small tree sizes, I got a file of hundreds of megabytes.

Check if you've set oob_score=True. For large training datasets this can result in a large matrix in oob_decision_function_. I kept the oob_score_, but deleted this matrix. Alternatively, you can set it to False.

Answer 4

I ran into a similar issue and was surprised to find out that indeed decision trees can easily take a lot of memory (range of MBs) and random forests will easily multiply that in the GB range. Details here: https://stackoverflow.com/a/72633003/4178189

See also some other consideration in this other answer: https://stackoverflow.com/a/72701704/4178189