39 votes
Accepted

Ways to deal with longitude/latitude feature

Lat long coordinates have a problem that they are 2 features that represent a three dimensional space. This means that the long coordinate goes all around, which means the two most extreme values are ...
Jan van der Vegt's user avatar
24 votes
Accepted

Should one hot vectors be scaled with numerical attributes

Once converted to numerical form, models don't respond differently to columns of one-hot-encoded than they do to any other numerical data. So there is a clear precedent to normalise the {0,1} values ...
Neil Slater's user avatar
  • 28.6k
22 votes

Feature Transformation on Input data

We love the normal form In most cases we try to make them act like normal. Its not classifiers point of view but its feature extraction view! Which Transformation? The main criterion in choosing a ...
Hadi Gharibi's user avatar
20 votes
Accepted

Why do we convert skewed data into a normal distribution

You might want to interpret your coefficients. That is, to be able to say things like "if I increase my variable $X_1$ by 1, then, on average and all else being equal, $Y$ should increase by $\beta_1$"...
Ricardo Magalhães Cruz's user avatar
17 votes
Accepted

Data scaling before or after PCA

I once heard a data scinetist state at a conference talk: "Basically, you can do what you want, as long as you know what you are doing." This also applies here. The more statistically sound way ...
Alex2006's user avatar
  • 309
13 votes

When should I use StandardScaler and when MinMaxScaler?

StandardScaler and MinMaxScaler are more common when dealing with continuous numerical data. One possible preprocessing approach for OneHotEncoding scaling is "soft-binarizing" the dummy ...
UrbanoFonseca's user avatar
12 votes

Zero Mean and Unit Variance

The questions of whether and why it's important, depends on the context. For gradient boosted decision trees, for example, it is not important - these ML algorithms "don't care" about monotone ...
Ami Tavory's user avatar
  • 1,227
11 votes

When should I use StandardScaler and when MinMaxScaler?

In "Python Machine Learning" by Raschka the author provides some guidance on page 111 when to normalize (min-max scale) and when to standardize data: Although normalization via min-max ...
Jonathan's user avatar
  • 5,360
9 votes
Accepted

Should I rescale tfidf features?

The most accepted idea is that bag-of-words, Tf-Idf and other transformations should be left as is. According to some: Standardization of categorical variables might be not natural. Neither is ...
wacax's user avatar
  • 3,390
8 votes
Accepted

How to give a higher importance to certain features in a (k-means) clustering model?

You cannot really use k-means clustering if your data contains categorical variables since k-means uses Euclidian distance which will not make a lot of sense with categorical variables. Check out the ...
georg-un's user avatar
  • 1,221
7 votes
Accepted

When should I NOT scale features

Scaling often assumes you know the min/max or mean/standard deviation, so directly scaling features where these information is not really known, can be a bad idea. For example, clipped signals may ...
Bruno Lubascher's user avatar
6 votes

Normalized output of machine learning

First you do not always need to normalize (standardize) the input vectors (feature vectors), sometimes is good, sometimes is bad. In general you scale your feature vector when the magnitude of a ...
Dani Mesejo's user avatar
  • 2,226
6 votes
Accepted

Do Clustering algorithms need feature scaling in the pre-processing stage?

Clustering algorithms are certainly effected by the feature scaling. Example: Let's say that you have two features: weight (in Lbs) height (in Feet) ... and we are using these to predict whether ...
Prashant's user avatar
6 votes

Linear Regression and scaling of data

You can't really talk about significance in this case without standard errors; they scale with the variables and coefficients. Further, each coefficient is conditional on the other variables in the ...
cactus_pardner's user avatar
6 votes

Data scaling before PCA: how to deal with categorical values?

You can not use PCA, or at least it is not recommended, for mixed data. It is best to use Factor analysis of mixed data. You are lucky that Prince is a Python package that covers all data scenarios, ...
TwinPenguins's user avatar
  • 4,207
6 votes
Accepted

What are some situations when normalizing input data to zero mean, unit variance is not appropriate or not beneficial?

A detailed answer to the question can be found here. [...]are there times when it is not appropriate or not beneficial? Short answer: Yes and No. Yes in the terms, that it can significantly ...
André's user avatar
  • 458
6 votes

When do we scale features and should it be done to label encoded features?

You should not use Label Encoding for Categorical data unless there is a known ranking and that also in the specified ratio between the level values. In this case, the model will assume 10 as 2 times ...
10xAI's user avatar
  • 5,544
6 votes
Accepted

feature scaling xgbRegressor

You're also scaling $y$, then of course you are getting lower error. That question was regarding scaling $X$. The same model will have very different error metrics when units on $y$ are changed: if I ...
David Masip's user avatar
  • 6,031
5 votes

Do Clustering algorithms need feature scaling in the pre-processing stage?

Yes. Clustering algorithms such as K-means do need feature scaling before they are fed to the algo. Since, clustering techniques use Euclidean Distance to form the cohorts, it will be wise e.g to ...
Sumit Asthana's user avatar
5 votes
Accepted

How to use the same minmaxscaler used on the training data with new data?

You are refitting scaler_x on your test set, which you don't want. Change this line: xaa = scaler_x.fit_transform(xaa) to <...
Bert Kellerman's user avatar
5 votes
Accepted

Is it better to use a MinMax or a Log Return normalization to predict stock price movements?

Log returns are symmetric compared to percentage change. log(a/b) = - log(b/a) and this (less skewness), in theory, leads to better results for most models (linear ...
keiv.fly's user avatar
  • 1,239
5 votes

How to handle features with very broad range

I suggest to try a log transformation. This has two potential benefits: The range of x values becomes smaller Your transformed data might be closer to resemble a normal distribution (only relevant ...
Jonathan's user avatar
  • 5,360
5 votes
Accepted

How does scaling affect Logistic Regression?

It affects anything optimized by a form of gradient descent, because it affects the relative scale of the dimensions of the input. If A is generally 1000x larger than B, then changing B's coefficient ...
Sean Owen's user avatar
  • 6,595
5 votes
Accepted

Does the performance of GBM methods profit from feature scaling?

Scaling doesn't affect the performance of any tree-based method, not for lightgbm, xgboost, catboost or even a decision tree. This post that elaborates on the topic, but mainly the issue is that ...
David Masip's user avatar
  • 6,031
5 votes

Model performance worsens after Cross Validation

With only 210 samples the difference might be caused by your train and test data not being from the same underlying distribution. That is, using holdout-CV to estimate model performance on such a ...
Jonathan's user avatar
  • 5,360
4 votes
Accepted

MinMaxScaler broadcast shapes

You made some mistakes on MinMaxScaler. MinMaxScaler shouldn't be fitted twice(as internal parameters inside ...
Icyblade's user avatar
  • 4,296
4 votes
Accepted

PCA and maintaining relationship with target variable

The short answer is that the y_original and x_reduced are still connected to each other, so it is safe to train your data using y_original and x_reduced. While x_reduced is on a different scale, as ...
TBSRounder's user avatar
4 votes

Do Clustering algorithms need feature scaling in the pre-processing stage?

In fact, most clustering algorithms are even highly sensitive to scaling. Rescaling the data can completely ruin the results. Bad scaling also appears to be a key reason why people fail with finding ...
Has QUIT--Anony-Mousse's user avatar
4 votes
Accepted

Linear Regression and scaling of data

The fact that the coefficients of hp and disp are low when data is unscaled and high when data are scaled means that these variables help explaining the dependent variable but their magnitude is large,...
David Masip's user avatar
  • 6,031
4 votes
Accepted

How to Normalize & Scale a Single Data Point

You need to normalise the input in the same way that the training data was normalised -- however, you don't need access to this training data during predictions of new data. If you have used a ...
timleathart's user avatar
  • 3,920

Only top scored, non community-wiki answers of a minimum length are eligible