1
$\begingroup$

I want to serve a sklearn model in server, any suggestions what is best format/method to save sklearn models. Currently I am using pickle.dump method to save the model.

$\endgroup$

2 Answers 2

0
$\begingroup$

According to the docs:

It is possible to save a model in scikit-learn by using Python’s built-in persistence model, namely pickle [...] In the specific case of scikit-learn, it may be better to use joblib’s replacement of pickle (dump & load), which is more efficient on objects that carry large numpy arrays internally as is often the case for fitted scikit-learn estimators, but can only pickle to the disk and not to a string [...]

I usually use joblib as it is very practical.

from joblib import dump, load
dump(clf, 'filename.joblib') 

clf = load('filename.joblib') 

As of Version 1.0.2. there is an additional remark on "interoperable formats":

For reproducibility and quality control needs, when different architectures and environments should be taken into account, exporting the model in Open Neural Network Exchange format or Predictive Model Markup Language (PMML) format might be a better approach than using pickle alone. These are helpful where you may want to use your model for prediction in a different environment from where the model was trained [...]

So in case you want to be flexible, you should look into ONNX or PMML.

$\endgroup$
3
  • $\begingroup$ Is it better to use ONNX or PMML format rather than joblib format? $\endgroup$
    – Taksh
    Commented Feb 25, 2022 at 15:09
  • $\begingroup$ In case you have no reason to believe that you will leave the Python/sklearn framework, there is no need for these tools. They are a „general“ framework to share models in different packages. I don‘t use them by now. I use joblib which is clean and easy $\endgroup$
    – Peter
    Commented Feb 25, 2022 at 15:31
  • $\begingroup$ With joblib you can also save scalers and transformers. Also very handy! $\endgroup$
    – Peter
    Commented Feb 25, 2022 at 15:32
0
$\begingroup$

Have you looked at the scikit-learn documentation, specifically the page on model persistence? They mention pickle as an option, but also note that using joblib may be better because of efficiency on larger numpy arrays.

$\endgroup$
1
  • $\begingroup$ Yes I did. I just need an expert opinion on that. I have zero experience so I wanted to explore some other options too. I also heard about ONNX too. $\endgroup$
    – Taksh
    Commented Feb 25, 2022 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.