How to scale prediction back after preprocessing

Question

So I'm a newbie to machine learning and am currently using the iris data set. I ran through a quick online tutorial about predicting stock prices and thought I'd try and do the iris one myself.

The issue I'm having is that I'm using preprocessing to scale the data to train my classifier. However when I make a prediction, the answer is also scaled. When I comment out all the preprocessing, I get accurate results. Is there a way to scale the prediction back?

The outputs are rounded to 0, 1 or 2 with each number representing one of three species.

You can see my code below:

import pandas as pd
import numpy as np
from sklearn import preprocessing, model_selection
from sklearn.linear_model import LinearRegression

df = pd.read_csv("iris.csv")

# setosa - 0
# versicolor - 1
# virginica - 2
df = df.replace("setosa", 0)
df = df.replace("versicolor", 1)
df = df.replace("virginica", 2)

X = np.array(df.drop(['species'], 1))

y = np.array(df['species'])

# Scale features
# X = preprocessing.scale(X)

X_train, X_test, y_train, y_test = model_selection.train_test_split(X, y, test_size=0.2)

clf = LinearRegression(n_jobs=1)  # Linear regression clf

clf.fit(X_train, y_train)

confidence = clf.score(X_test, y_test)

print("Confidence: " + confidence)

# Inputs
sepal_length = float(input("Enter sepal length: "))
sepal_width = float(input("Enter sepal width: "))
petal_length = float(input("Enter petal length: "))
petal_width = float(input("Enter petal width: "))

# Create panda data frame with inputted data
index = [0]
d = {'sepal_length': sepal_length, 'sepal_width': sepal_width, 'petal_length': petal_length, 'petal_width': petal_width}
predict_df = pd.DataFrame(data=d, index=index)

# Create np array of features
predict_X = np.array(predict_df)

# Need to scale new X feature values
# predict_X = preprocessing.scale(predict_X, axis=1)

# Make a prediction against prediction features
prediction = clf.predict(predict_X)

print(predict_X, prediction)

rounded_prediction = int(round(prediction[0]))

if rounded_prediction == 0:
    print("== Predicted as Setosa ==")
elif rounded_prediction == 1:
    print("== Predicted as Versicolor ==")
elif rounded_prediction == 2:
    print("== Predicted as Virginica ==")
else:
    print("== Unable to make a prediction ==")

Here is an example of my output with preprocessing enabled. I'll be using one of the lines from the CSV as an example (6.4 sepal length, 3.2 sepal width, 4.5 petal length and 1.5 petal width) which should equal the versicolor species (1):

Confidence: 0.9449475378336242
Enter sepal length: 6.4
Enter sepal width: 3.2
Enter petal length: 4.5
Enter petal width: 1.5
[[ 1.39427847 -0.39039797  0.33462683 -1.33850733]] [0.41069281]
== Predicted as Setosa ==

Now with preprocessing commented out:

Confidence: 0.9132522144785978
Enter sepal length: 6.4
Enter sepal width: 3.2
Enter petal length: 4.5
Enter petal width: 1.5
[[6.4 3.2 4.5 1.5]] [1.29119283]
== Predicted as Versicolor ==

It seems I'm either doing the preprocessing wrong, or there's an extra step that I've missed out. I'm sorry if I get some of the terminology wrong and thanks in advance for answering.

Answer 1

I think your methodology is correct, but this line:

# Scale features
# X = preprocessing.scale(X)

should be changed to:

# Scale features
# X = preprocessing.scale(X, axis = 1)

As the default for scale is to set axis to 0 (I wonder why!). If the problem persists comment it and I will edit.

Edit

Although your methodology is not wrong, it is more suitable to use sklearn StandardScaler. See the documentation of this class. Usually, it is better to fit the scaler with the training data and transform the test data according to that fit.

Answer 2

When you decided you have to scale your data, you usually have to follow these steps:

For training:

1. Scale / Standarize the training set
2. Store the scaling / standarization factors of the training set
3. Train the model

For predicting:

1. Scale / standarize the input data, but very important, with the scaling/standarization factors stored during the training process. You dont have to compute the min, max or mean values of the new data.
2. Predict

The reason is that you have to map the new data to the same feature-space used for the training process, so you have to scale/std it with the same factors, otherwise, you are changing the feature space.