What are the best practices to save, store, and share machine learning models?
In Python, we generally store the binary representation of the model, using pickle or joblib. Models, in my case, can be ~100Mo large. Also, joblib can save one model to multiple files unless you set
But then, if you want to control access rights to models, and be able to use models from different machines, what's the best way to store them?
I have a few choices:
- Store them as files, and then put them in a repository using Git LFS
- Store them in an SQL database as binary files:
- For instance in Postgresql https://wiki.postgresql.org/wiki/BinaryFilesInDB
- This is also the method recommended by the SQL Server team: