29
$\begingroup$

What are the best practices to save, store, and share machine learning models?

In Python, we generally store the binary representation of the model, using pickle or joblib. Models, in my case, can be ~100Mo large. Also, joblib can save one model to multiple files unless you set compress=1 (https://stackoverflow.com/questions/33497314/sklearn-dumping-model-using-joblib-dumps-multiple-files-which-one-is-the-corre).

But then, if you want to control access rights to models, and be able to use models from different machines, what's the best way to store them?

I have a few choices:

$\endgroup$
  • $\begingroup$ Did you get an efficient way to do this? $\endgroup$ – iNet Jun 3 '19 at 15:54
6
$\begingroup$

You may have a look at nexus or dvc or datmo.

There was recently a presentation at at meetup in berlin, zalandos AI data engineering meetup.

| improve this answer | |
$\endgroup$
5
$\begingroup$

I faced this problem (and still face it today) for many years. I really thing that, if you don't provide detailed requirements, you can't expect a serious answer. I explain myself with examples of my work:

  • I regularly try multiple variations of the same model to find what parameters work best. It takes several day to train one single model which produces some output that is later used for evaluation. To do so, I make a simple NumPy dump of the model since it is easy to share it between servers, or colleagues. You should avoid pickle since it stores much more (instances of class, libraries...) than just the parameters learned by your model. Importing the model on another machine might not work if the python environment slightly differs.

  • When pushing a model in production, I need 1) a version of the model that I can load fast in case of a server breakdown (typically a binary format, storing only what is necessary such as weights of a neural network) and 2) a way to keep the model in-RAM to quickly deal with the API requests.

For two different purposes, I need three different formats. Then, more generally speaking, the choice of the format depends on the tools you use. For example, if you work with TensorFlow, you might be interested in their TensorFlow Serving system

| improve this answer | |
$\endgroup$
5
$\begingroup$

I would like to suggest 2 more approaches.

  1. Store them in document storage (eg. mongoDB) - this method is recommended when your model files are less then 16Mb (or the joblib shards are), then you can store model as binary data. in addition, some ML libraries support model export and import in json (eg. LightGBM), which makes it a perfect candidate for storage in document storage. Advantages: easy tracking of model generation and easy access, Disadvantages: things will get messy if model object is too large.

  2. Store your model on object storage (eg. Amazon S3) - this method is good if your models are very large, in this case you get unlimited storage and fairly easy API, you pay more, that is for sure. Advantages: Unlimited space and ability to store arbitrary file formats. Disadvantages: cost, and the fact that to do it right you'll need to develop your own tracking system.

good luck!

| improve this answer | |
$\endgroup$
0
$\begingroup$

I have faced this issue for many years, as a data scientist or an ML engg. we have to create a ton of models before coming to a conclusion. Effectively storing all the model's profiles, parameters, and features used is a pain specially if you have multiple notebooks.

modellogger is a python package that can help you to organise the stuffs and create a full blown summary with dynamic stats on the call of an function.

pip install modellogger

follow the 3 steps easy documentation and voila you have it.

| improve this answer | |
New contributor
Kaiser is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.