I have trained an ANN model for a regression problem which takes 10 parameters as input and gives 1 output. After training, I saved the model as json and weights as a .h5 file using keras. Now I want to make predictions on the new data. I have loaded the model and my question here is how do I scale this single row of input values, before feeding it to the model? Some of the machine learning algorithms did not require scaling(Standardization/Normalization) So I could load those models and used for making predictions. How do I do in an ANN model, since we scale the data for training?

  • $\begingroup$ What did you use for scaling? If you used sklearn's library, you could simply save the scaler object as a pickle object and further use it while testing. $\endgroup$ – bkshi Jun 24 '19 at 12:20
  • $\begingroup$ yes, I have used sklearn library. So along with model and weights, scaler should also be saved. Is my understanding correct? Is this how I should do when I deploy my model also. $\endgroup$ – cvg Jun 24 '19 at 12:28
  • 1
    $\begingroup$ Yes, you are right. At the time of deployment also that scaler should be used as it is trained on the training data. $\endgroup$ – bkshi Jun 24 '19 at 12:39

Use sklearn.preprocessing.OneHotEncoder for example and transfer the one-hot encoding to your web-service ( i'm guessing that's how you're using the model for inference ) via sklearn.pipeline.Pipeline. The pipeline will save the state of your fit on your training data and apply the same function on your production data.

Example :

pipeline1 = Pipeline([
                ('OneHotEncoder', OneHotEncoder())

This is how you create a pipeline containing the onehotencoder , fit your data on the pipeline. All is left is dumping your pipeline in a file, loading it later in your production environment, and call the transform method on your loaded pipeline :

# Production environment
pipeline1 = joblib.load('pipeline1.joblib')
momo = pipeline1.transform(productiondata.column1.values.reshape(-1,1)).toarray()

And here , the variable momo contains your production data with the pipeline ( containing the one-hot encoding operation ) applied to it.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.