Skip to main content
Data Science

Return to Question

Became Hot Network Question
deleted 289 characters in body
Source Link
Dave
Dave
  • 248
  • 2
  • 7

We know that the best practice in data preprocessing (such as standardization, Normalization, ... etc) is that while we perform fit_trasform() on the training data, we apply transform() testing data so that the learned parameters from scaling the train data are applied on testing data. Similar to this:

from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform (X_test)

The question is: Does it also make sense to perform fit_transform() on the training data but NOT transform() testing data at all so that we get to test the model performance on actual real-world data that are not transformed at all? In fact, I tested this case using scikit-learn library for StandardScaler before posting this question but I get an error so I thought this may not be an acceptable case to do, hence we always need to transform() test data if we apply any preprocessing technique on training data.

Thank you

We know that the best practice in data preprocessing (such as standardization, Normalization, ... etc) is that while we perform fit_trasform() on the training data, we apply transform() testing data so that the learned parameters from scaling the train data are applied on testing data. Similar to this:

from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform (X_test)

The question is: Does it also make sense to perform fit_transform() on the training data but NOT transform() testing data at all so that we get to test the model performance on actual real-world data that are not transformed at all? In fact, I tested this case using scikit-learn library for StandardScaler before posting this question but I get an error so I thought this may not be an acceptable case to do, hence we always need to transform() test data if we apply any preprocessing technique on training data.

Thank you

We know that the best practice in data preprocessing (such as standardization, Normalization, ... etc) is that while we perform fit_trasform() on the training data, we apply transform() testing data so that the learned parameters from scaling the train data are applied on testing data. Similar to this:

from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform (X_test)

The question is: Does it also make sense to perform fit_transform() on the training data but NOT transform() testing data at all so that we get to test the model performance on actual real-world data that are not transformed at all?

Thank you

Source Link
Dave
Dave
  • 248
  • 2
  • 7

Is it acceptable not to transform() test data after train data is being fit_transform()-ed

We know that the best practice in data preprocessing (such as standardization, Normalization, ... etc) is that while we perform fit_trasform() on the training data, we apply transform() testing data so that the learned parameters from scaling the train data are applied on testing data. Similar to this:

from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform (X_test)

The question is: Does it also make sense to perform fit_transform() on the training data but NOT transform() testing data at all so that we get to test the model performance on actual real-world data that are not transformed at all? In fact, I tested this case using scikit-learn library for StandardScaler before posting this question but I get an error so I thought this may not be an acceptable case to do, hence we always need to transform() test data if we apply any preprocessing technique on training data.

Thank you