0
$\begingroup$

I've heard that we should use train dataset's scale for that of test data so they are in line with each other in terms of scale.

And I know we use transform() function for the test data preprocessing.

But I am wondering how I can apply the same logic to the real-life data set after we successfully save and load a model since I don't think we can use transform() function anymore.

From what I understand, there must be a way to "extract" mean and standard deviation from the train data set and to apply it for real-life model when using the model we saved already, but how can we do that?

$\endgroup$

2 Answers 2

1
$\begingroup$

Ideally, the transform operation is part of your pipeline, therefore, if you have reallife data, with the same pipeline, it will apply the same transformation.

(I'm assuming you're using a modeling language that makes use of pipelines)

$\endgroup$
0
$\begingroup$

It has to be saved somewhere i.e. Database after the training is done.
The saved values should be used on new data and all these steps should work in a loop i.e. when you re-train again the values will be updated and saved again.

e.g. If we see the Keras pre-trained models, it provides the necessary pre-processing function. We can directly use that

from keras.applications.resnet50 import preprocess_input
train_datagen = ImageDataGenerator( preprocessing_function=preprocess_input)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.