When training a model we split the dataset into training set and test set. In case a normalization/standardization is needed on any column then this process is done separately for training set and test set to prevent data leakage. The normalization of the test set is done on the parameters of training set. The code that we usually write is something like below -
scaler = StandardScaler() #scaler = MinMaxScaler() train = scaler.fit_transform(train) test = scaler.transform(test)
fit part figures out the parameters and then
transform part does the normalization.
Now my question is - When the model is deployed in production, the column from real life data also needs to be normalized. How do we do that? Do we store this
scaler somewhere (may be pickle) or do we store the parameters of the
scaler (like min, max, mean, standard deviation etc.) somewhere (e.g. a file, table) and then read the stored file or table to retrieve the values of parameters and perform normalization of the new real life data before calling the