I have a Support Vector Machine in Scikit-learn (Python) that gets trained once in a while when enough new data has accumulated (user help train the model by submitting new data).

I store the model in .pkl format for persistence. However, the SVM needs scaled data and I'm wondering what would be a good way to persist the scaler, since it was fitted on the train data that changes over time (as users submit more data).

Is there a common solution for this? Or is there a value that I can store with the model that works as some sort of seed to recreate the scaler?

EDIT (added implementation):

I have a database with a dataset. users periodically add data points to this dataset. I also have an estimator database which keeps track of all the trained models and their .pkl file locations. When enough new data points are added the system will start to train a new estimator. It extracts all the features and scales the data, then trains a model. After that it compare the existing best model with the newly trained one and if the new one is better, it sets the new one to 'active'. There is only one active model a a time and that is the one that will be used for predictions on new data. (I hope this helps)

  • $\begingroup$ @StephenRauch I added it, see the edit above. $\endgroup$ – NG. Dec 19 '17 at 14:23
  • $\begingroup$ Can you explicity tell what language you are working with, please? $\endgroup$ – tagoma Dec 19 '17 at 21:53
  • $\begingroup$ my bad! I'm working with Scikit-learn in Python $\endgroup$ – NG. Dec 20 '17 at 9:38

So after looking some more I found this question: save scaler model in sklearn which says that one can picle the scaler aswell, which works.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.