How to Normalize & Scale a Single Data Point

Question

I do understand the concept of normalizing & scaling the training/test data; it does help with the converging of the cost function. It is a great helper for many of the machine learning algorithms.

• I train and validate my model with the normalized data (MinMaxScaler) and save my model.
• A new input data comes in and I want to use my saved model to make predictions.
• I no longer have access to the training/test data at this point. All I have is the new single row of input data.
• How will I normalize this single vector of input data so that it can be fed to my model?

The only normalization that I can think of simply linear transformation of data (e.g. simply map values from [40, 70] range to [-1, 1]). I need a normalization technique that doesn't depend on the full range of data.

Thanks!

Answer 1

You need to normalise the input in the same way that the training data was normalised -- however, you don't need access to this training data during predictions of new data. If you have used a MinMaxScaler for example, then you can re-use this to transform your new data point:

scaler = MinMaxScaler()
X_normalised = scaler.fit_transform(X)

# do other stuff here like train & validate your model

# now we have a new test data point from somewhere, we scale
# it using the scaler we fitted on the training data
new_test_point_normalised = scaler.transform(new_test_point)

# now we can classify it with our model!

You can serialise the scaler to disk using pickle and re-load it when predicting new data points if necessary (i.e., the model training and testing happen in two different scripts).