When saved to disk using cPickle: https://stackoverflow.com/questions/20662023/save-python-random-forest-model-to-file, my random forest is 6.57 GB.
with open('rforest.cpickle', 'wb') as f:
cPickle.dump(rforest, f)
I want to use the forest itself to make predictions via a python API hosted on Heroku -- of course, that file size is unacceptable.
Why is the file size so large? There are 500 trees in the ensemble -- all I want to save are the completed trees themselves, since they will be used as prediction. Is it the actual nodes and edges that compose each of the 500 trees that requires nearly 7 GB of space on disk?
I used scikitlearn's randomforestregressor:
def buildForest(self, X_train, y_train):
rf = RandomForestRegressor(n_estimators=500, verbose=1)
rf.fit_transform(X_train, y_train)
return rf
Also, if there is a better way to make my model accessible via API, that would also be good to know.
Update: I reduced it to 100 trees without losing much predictive power, and so now the saved size is 1.3 GB -- much more manageable, but still not great.