Questions tagged [feature-scaling]

Feature scaling is a data pre-processing step where the range of variable values is standardized. Standardization of datasets is a common requirement for many machine learning algorithms. Popular feature scaling types include scaling the data to have zero mean and unit variance, and scaling the data between a given minimum and maximum value.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Which Frameworks/Libs Best Support Integer-Based Features, Scaling, Training, etc?

Papers such as Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference have interested me in exploring integer-based data science. In particular, I'm thinking of ...
ezekiel68's user avatar
0 votes
0 answers
11 views

Best distance metric and estadarization method for clustering with percentages data

I'm studying access patterns to a facility with clustering. My variables are percentages. For example, for each user, I have the percentage of access 'in time' versus late, or the percentage of using ...
Kaikus's user avatar
  • 101
1 vote
0 answers
12 views

Clustering Similar Articles Using Mixed Data: Seeking Advice and Validation

Question: I'm working on a project where I need to cluster a dataset of articles based on various features, including text, numeric values, and categorical data. I've implemented a clustering approach ...
sara sara's user avatar
0 votes
2 answers
37 views

Scaling of categorical feautres

In the context of algorithms that consider the scales of features, I have a situation where some features are encoded using ordinal encoding, some features are binary, and some features are standard ...
jxqbbb's user avatar
  • 51
0 votes
1 answer
27 views

Different scaling methods of different features results in a faux dependency between them

My dataset contains the following two features: "movie duration" (minutes) and "tv shows duration" (seasons). If a certain sample is of type "movie", it's duration will ...
Shir's user avatar
  • 1
0 votes
1 answer
12 views

feature engineering mechanism

why do we need to rescale some feature having large range I know we do it for faster rate of gradient descent ,but still how does rescaling works? and it doesn't break the model and does rescaling ...
rushi jhala's user avatar
1 vote
4 answers
60 views

Are scalers or encoders supposed to be serialized along with trained models?

Consider the very basic example below: ...
Muhammad Usman's user avatar
1 vote
1 answer
20 views

When training a sklearn machine learning model, what part of a data from a csv file needs scaling like MaxAbsScaler or MinMaxScaler?

Consider the code below: ...
Muhammad Usman's user avatar
0 votes
0 answers
7 views

Feature Selection using Statistical Testing on Features with Different Scales

If I want to select features based on ANOVA test for example, should I scale features to same/similar range so the results can be compared , or is it unnecessary?
jxqbbb's user avatar
  • 51
0 votes
1 answer
33 views

Standard Scaling not resulting in good nn inputs

I am using standard scaling to transform my dataset for input to a NN. The normalized dataset is in the range +-0.04. I am getting different results in the NN when simply multiplying by a factor of ...
BAR's user avatar
  • 101
5 votes
2 answers
384 views

How to handle similarity search on mixed data types vectors?

I think this question is one that many beginners run into and I could not find a decent generic guide for it. My issue is the following. I want to evaluate similarity of vectors which have mixed data ...
Chapo's user avatar
  • 53
1 vote
0 answers
55 views

How to encode & scale IP addresses as input for ML models

Im currently working on an anomaly detection while making a transaction. As a part of the data that I extracted, I have the IP addresses of the indivduals who made the transaction. Since the IP ...
Sivadithiyan official's user avatar
1 vote
1 answer
251 views

StandardScaler and MinMaxScaler vs RobustScaler

I've recently read that Standard Scaler functions best in situations where the distribution of the features are approximately normal. MinMaxScaler works in a way that it preserves the features' ...
Arthur Caldas's user avatar
0 votes
0 answers
33 views

Logistic Regression, Standardization, Stationarity, Differencing

I am going to be using the logistic regression in which I will use L2 Regularization. I have these 4 rolling standard deviation variables. Here are the results of the Augmented Dickey-Fuller Test for ...
DomIsAwesomee's user avatar
0 votes
0 answers
18 views

Scaling nominal vars K means

I had a discussion recently with a coworker. We are running a K means clustering algorithm. He said that when dummy variables are made into 1s and 0s,these columns must be scaled in a specific way. If ...
NKI's user avatar
  • 1
0 votes
0 answers
11 views

Clustering methods for text and image features

I want to build a recommender system with unlabeled data and used TF-IDF to extract text features from a given short description and VGG-16 to extract image features. I am looking for a way to combine ...
Dan G's user avatar
  • 1
0 votes
1 answer
179 views

Combining text and image features with different scales

I have computed text features using [SBERT][1] and image features using VGG-16. The text features range from -1.58 to 1.58, whereas the image features range between 0 and 521. I would want to ...
Dan G's user avatar
  • 1
0 votes
1 answer
36 views

How to treat categorical columns after ordinal encoding?

If encode three categorical variables like "bad", "normal", "good" into 0,1,2, after that can I treat them as numerical values? So can I perform on them MinMaxScaler or ...
Flavio Brienza's user avatar
0 votes
0 answers
25 views

Scaling/Normalization again

I would like to ask for normalization .. again. I'm working on LSTM model having a 100x4 dataset used for timeseries prediction. My question is on which set of data the scalar/normalizer should be run ...
Paul Paku's user avatar
  • 101
1 vote
2 answers
247 views

problem with standardScaler

problem with standardScaler hi I'd like to scale one column in the titanic data set. I am using the following code segment. for some reason df_scaled results an empty set. how can I solve it? what is ...
Mehmet Deniz's user avatar
1 vote
1 answer
61 views

Customer Segmentation with mixed data [closed]

I want to perform clustering. I am reading about this topic but I am totally confused. My dataset has 490 observations and it consists of numerical data (3 columns: Recency, Frequency, Monetary), ...
ArgyGr's user avatar
  • 11
1 vote
0 answers
160 views

Is there a need to use scaling for Age attribute?

What is the good way working with 'Age' attribute? Don't touch it or should it be scaled? Below photo shows my results 'Before' and 'After' standardization.
hyper-cookie's user avatar
0 votes
0 answers
15 views

How to Forecast Stock Index against Global Indicators

My goal is predict the benchmark index of the Indian stock market with the help machine learning algorithm using of global market indices as mentioned below. Put simply, forecast whether tomorrow’s ...
Venkatesh Kuppusamy's user avatar
0 votes
0 answers
24 views

Is it sensible to use conventional scaling methods to scale features that have meaningful (+/-)sign?

I'm working on a project where the input features contain velocities of objects. The sign in the velocity feature is meaningful i.e. negative velocity means an object moving to the left while positive ...
Eman.suradi's user avatar
0 votes
1 answer
32 views

Success metric of database migration using row counts

Description I have a problem where I'm tasked to successfully transform and repurpose data from one SQL server to another. Call the source $\text{src}$ and the target database $\text{tgt}$. In order ...
ArnoV's user avatar
  • 101
1 vote
1 answer
74 views

Is it valid to use Spark's StandardScaler on sparse input?

While I know it's possible to use StandardScaler on a SparseVector column, I wonder now if this is a valid transformation. My reason is that the output (most likely) will not be sparse. For example, ...
user12138762's user avatar
1 vote
1 answer
144 views

Inverse Scaling Partitioned Data

I have scaled an original matrix A with sklearn's StandardScaler, resulting to a matrix S. I then partitioned the result into ...
ccccc's user avatar
  • 11
0 votes
0 answers
48 views

What does MinMax scaler do poorly for Ridge regression?

I tried ridge regression on a dataset scaled using min-max scaler to perform binary classification, and all predictions were made to a single class. Both the training dataset and the test set were ...
ron burgundy's user avatar
0 votes
2 answers
86 views

Understanding the relationship between two features

I am trying to solve classification task. Could you suggest me, if these two features are independent? the plot looks strange
Любовь Пономарева's user avatar
0 votes
1 answer
468 views

Scaling the target variables for Neural Networks in Regression Problem

I am trying to implement a neural network on a regression problem. I have scaled the independent variables since this is a crucial step for neural networks. However, I see online that some people ...
John adams's user avatar
0 votes
1 answer
141 views

Standardization after log transformation

I have a few question about log transformation and standardization. First: Should I standardize my features after doing log transformation? Second: I still do not understand, because when doing log ...
Jovian Aditya's user avatar
0 votes
1 answer
139 views

How to scale a subset of data with respect to the entire dataset

I am developing a financial time-series prediction model using sklearn using StandardScaler for scaling purposes. I train a model, and then use the model regularly ...
functorial's user avatar
0 votes
2 answers
526 views

Should I split data into train/validation/test before feature scaling and feature selection or after?

I'm working on a project, I finished data preprocessing, and I found an article where it says that feature scaling and feature selection should be done after splitting data, some other articles say it ...
biihu's user avatar
  • 11
0 votes
1 answer
345 views

Feature scaling in Linear Regression

I always use Linearregression() class in sklearn library for creating a linear regression model. According to my understanding, we need feature scaling in linear ...
AAA's user avatar
  • 35
2 votes
1 answer
1k views

Predict actual result after model trained with MinMaxScaler LinearRegression

I was doing the modeling on the House Pricing dataset. My target is to get the mse result and predict with the input variable I have done the modeling, I'm doing the modeling with scaling the data ...
MADFROST's user avatar
  • 123
3 votes
1 answer
72 views

What justifies feature scaling?

Although I can understand the significance of feature scaling in some cases (e.g. when gradient descent is involved), I don't feel I understand the necessity of this process in general. But there a ...
ado sar's user avatar
  • 181
2 votes
0 answers
27 views

When is scaling and centering important?

There are some models such as PCA or SVM where scaling and centering of training data is essential. There are some models, mostly tree-based where scaling and centering is not required at all. I don't ...
xan's user avatar
  • 121
0 votes
1 answer
59 views

How to Scale target feature

How should I scale target feature? Should I use scaler as fit_transform on y_train, and just fit on y_test to avoid leaking data?
Rus Zzzeta's user avatar
2 votes
1 answer
44 views

Choosing Right Optimiser and Data Scaling

The choice of optimiser and how data is scaled are both very important things in machine learning, yet they are not hyperparameters (as far as I am aware). It is also not necessarily obvious which ...
Socorro's user avatar
  • 91
1 vote
2 answers
112 views

Ways of scaling scores on data without knowing possible maximum values

In my scenario, I have to process some input data and give a score based on what the processing phase outputs. The problem is that, in order to scale the score in a human-readable format I'd have to ...
Ionut-Alexandru Baltariu's user avatar
1 vote
1 answer
281 views

Is test data required to be transformed by training data statistics?

I am using a dataset (from literature) to build an MLP and classify real-world samples (from wetlab experiment) using this MLP. The performance of MLP on the literature dataset are well enough. I am ...
Apollonia Vitelli's user avatar
0 votes
0 answers
105 views

Do I need to encode numerical variables like "year"?

I have a simple time-series dataset. it has a date-time feature column. ...
smarks70's user avatar
0 votes
1 answer
71 views

Why does log-transforming the target have a huge impact on MSE value?

I am doing linear regression using the Boston Housing data set, and the effect of applying $\log(y)$ has a huge impact on the MSE. Failing to do it gives MSE=34.94 ...
Caterina's user avatar
  • 119
1 vote
1 answer
3k views

Feature selection before or after scaling and splitting

Should feature scaling/standardization/normalization be done before or after feature selection, and before or after data splitting? I am confused about the order in which the various pre-processing ...
Caterina's user avatar
  • 119
0 votes
2 answers
80 views

Standardization in combination with scaling

Would it be ok to standardize all the features that exhibit normal distribution (with StandardScaler) and then re-scale all the features in the range 0-1 (with <...
Caterina's user avatar
  • 119
2 votes
2 answers
235 views

Is it wise to always `StandardScaler()` features? [SOLVED]

My current investigations point to the sklearn.preprocessing.StandardScaler() not always being the right choice for certain types of feature extractions for neural ...
jake_asks_short_questions's user avatar
0 votes
1 answer
62 views

Scaled-input-trained ML Models

I have input data, the magnitudes vary quite a lot between features. I have scaled them using sklearn's StandardScaler(), then used keras to train a NN on this data to predict my target. I have ...
Socorro's user avatar
  • 91
0 votes
1 answer
5k views

__init__() takes 1 positional argument but 4 were given sklearn standard scaler error

I defined a class like below: ...
Mohsen Mahmoodzadeh's user avatar
0 votes
1 answer
127 views

Feature scaling on null values

How to handle null values in dataset for performing feature scaling on a particular column? i.e.Should we keep the null value as it is or impute some other value? Is there any tutorial on how to ...
Pratik Goud's user avatar
1 vote
0 answers
26 views

Feature Scaling + Selection when target is imbalanced

If my target is imbalanced, when should I do target balancing in preparation for modeling? Before feature scaling and selection? After feature scaling and selection? If I am doing backward elimination,...
Denisse's user avatar
  • 11

1
2 3 4 5 6