Should I standardize first or generate polynomials first?

Question

Recently I am dealing a classification problem with some algorithms, say logistic regression.

When I preprocess my data, I standardize all my features and then generate polynomial features based on them.

from sklearn.preprocessing import PolynomialFeatures, StandardScaler

and I do

# features is my entire features dataset, labels excluded 
features = poly.fit_transform(features)
features = std.fit_transform(features)

After finishing training my model, the accuracy is, say about 80%. Then I invert the two line of preprocessing code to

features = std.fit_transform(features)
features = poly.fit_transform(features)

I have read this post but it seems the answers is not strong enough to help me figure it out.

Should I standardize my data first or generate polynomials from original features first?

Answer 1

Depends on what are you using the standardization for and how the features relate to your problem, but I guess for most situations you’re better off standardizing them afterwards.

If you are looking for some standardization in order to make optimization of some algorithm faster (e.g. neural networks rather than logistic regression), it might be a better idea to standardize afterwards, as otherwise it defies the purpose.

If you are doing it in order to introduce some regularization, then standardizing them to 0-1 first will make the polynomials smaller overall and thus more likely to be given weights compared to the original features, which is probably not what you want.