Questions tagged [feature-scaling]
Feature scaling is a data pre-processing step where the range of variable values is standardized. Standardization of datasets is a common requirement for many machine learning algorithms. Popular feature scaling types include scaling the data to have zero mean and unit variance, and scaling the data between a given minimum and maximum value.
258
questions
0
votes
0
answers
8
views
Which Frameworks/Libs Best Support Integer-Based Features, Scaling, Training, etc?
Papers such as Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference have interested me in exploring integer-based data science. In particular, I'm thinking of ...
0
votes
0
answers
11
views
Best distance metric and estadarization method for clustering with percentages data
I'm studying access patterns to a facility with clustering.
My variables are percentages. For example, for each user, I have the percentage of access 'in time' versus late, or the percentage of using ...
- 101
1
vote
0
answers
12
views
Clustering Similar Articles Using Mixed Data: Seeking Advice and Validation
Question:
I'm working on a project where I need to cluster a dataset of articles based on various features, including text, numeric values, and categorical data. I've implemented a clustering approach ...
- 11
0
votes
2
answers
37
views
Scaling of categorical feautres
In the context of algorithms that consider the scales of features, I have a situation where some features are encoded using ordinal encoding, some features are binary, and some features are standard ...
- 51
0
votes
1
answer
27
views
Different scaling methods of different features results in a faux dependency between them
My dataset contains the following two features: "movie duration" (minutes) and "tv shows duration" (seasons). If a certain sample is of type "movie", it's duration will ...
- 1
0
votes
1
answer
12
views
feature engineering mechanism
why do we need to rescale some feature having large range I know we do it for faster rate of gradient descent ,but still how does rescaling works? and it doesn't break the model
and does rescaling ...
1
vote
4
answers
60
views
Are scalers or encoders supposed to be serialized along with trained models?
Consider the very basic example below:
...
1
vote
1
answer
20
views
0
votes
0
answers
7
views
Feature Selection using Statistical Testing on Features with Different Scales
If I want to select features based on ANOVA test for example, should I scale features to same/similar range so the results can be compared , or is it unnecessary?
- 51
0
votes
1
answer
33
views
Standard Scaling not resulting in good nn inputs
I am using standard scaling to transform my dataset for input to a NN. The normalized dataset is in the range +-0.04. I am getting different results in the NN when simply multiplying by a factor of ...
- 101
5
votes
2
answers
384
views
How to handle similarity search on mixed data types vectors?
I think this question is one that many beginners run into and I could not find a decent generic guide for it.
My issue is the following. I want to evaluate similarity of vectors which have mixed data ...
- 53
1
vote
0
answers
55
views
How to encode & scale IP addresses as input for ML models
Im currently working on an anomaly detection while making a transaction. As a part of the data that I extracted, I have the IP addresses of the indivduals who made the transaction. Since the IP ...
1
vote
1
answer
251
views
StandardScaler and MinMaxScaler vs RobustScaler
I've recently read that Standard Scaler functions best in situations where the distribution of the features are approximately normal.
MinMaxScaler works in a way that it preserves the features' ...
0
votes
0
answers
33
views
Logistic Regression, Standardization, Stationarity, Differencing
I am going to be using the logistic regression in which I will use L2 Regularization. I have these 4 rolling standard deviation variables. Here are the results of the Augmented Dickey-Fuller Test for ...
0
votes
0
answers
18
views
Scaling nominal vars K means
I had a discussion recently with a coworker. We are running a K means clustering algorithm. He said that when dummy variables are made into 1s and 0s,these columns must be scaled in a specific way. If ...
- 1
0
votes
0
answers
11
views
Clustering methods for text and image features
I want to build a recommender system with unlabeled data and used TF-IDF to extract text features from a given short description and VGG-16 to extract image features. I am looking for a way to combine ...
- 1
0
votes
1
answer
179
views
Combining text and image features with different scales
I have computed text features using [SBERT][1] and image features using VGG-16. The text features range from -1.58 to 1.58, whereas the image features range between 0 and 521. I would want to ...
- 1
0
votes
1
answer
36
views
How to treat categorical columns after ordinal encoding?
If encode three categorical variables like "bad", "normal", "good" into 0,1,2, after that can I treat them as numerical values? So can I perform on them MinMaxScaler or ...
0
votes
0
answers
25
views
Scaling/Normalization again
I would like to ask for normalization .. again.
I'm working on LSTM model having a 100x4 dataset used for timeseries prediction.
My question is on which set of data the scalar/normalizer should be run ...
- 101
1
vote
2
answers
247
views
problem with standardScaler
problem with standardScaler
hi
I'd like to scale one column in the titanic data set. I am using the following code segment.
for some reason df_scaled results an empty set.
how can I solve it? what is ...
- 31
1
vote
1
answer
61
views
Customer Segmentation with mixed data [closed]
I want to perform clustering. I am reading about this topic but I am totally confused. My dataset has 490 observations and it consists of numerical data (3 columns: Recency, Frequency, Monetary), ...
- 11
1
vote
0
answers
160
views
Is there a need to use scaling for Age attribute?
What is the good way working with 'Age' attribute? Don't touch it or should it be scaled?
Below photo shows my results 'Before' and 'After' standardization.
- 111
0
votes
0
answers
15
views
How to Forecast Stock Index against Global Indicators
My goal is predict the benchmark index of the Indian stock market with the help machine learning algorithm using of global market indices as mentioned below. Put simply, forecast whether tomorrow’s ...
0
votes
0
answers
24
views
Is it sensible to use conventional scaling methods to scale features that have meaningful (+/-)sign?
I'm working on a project where the input features contain velocities of objects. The sign in the velocity feature is meaningful i.e. negative velocity means an object moving to the left while positive ...
- 101
0
votes
1
answer
32
views
Success metric of database migration using row counts
Description
I have a problem where I'm tasked to successfully transform and repurpose data from one SQL server to another. Call the source $\text{src}$ and the target database $\text{tgt}$.
In order ...
- 101
1
vote
1
answer
74
views
Is it valid to use Spark's StandardScaler on sparse input?
While I know it's possible to use StandardScaler on a SparseVector column, I wonder now if this is a valid transformation. My reason is that the output (most likely) will not be sparse. For example, ...
- 13
1
vote
1
answer
144
views
Inverse Scaling Partitioned Data
I have scaled an original matrix A with sklearn's StandardScaler, resulting to a matrix S.
I then partitioned the result into ...
- 11
0
votes
0
answers
48
views
What does MinMax scaler do poorly for Ridge regression?
I tried ridge regression on a dataset scaled using min-max scaler to perform binary classification, and all predictions were made to a single class. Both the training dataset and the test set were ...
- 11
0
votes
2
answers
86
views
Understanding the relationship between two features
I am trying to solve classification task. Could you suggest me, if these two features are independent? the plot looks strange
0
votes
1
answer
468
views
Scaling the target variables for Neural Networks in Regression Problem
I am trying to implement a neural network on a regression problem. I have scaled the independent variables since this is a crucial step for neural networks. However, I see online that some people ...
- 221
0
votes
1
answer
141
views
Standardization after log transformation
I have a few question about log transformation and standardization.
First: Should I standardize my features after doing log transformation?
Second: I still do not understand, because when doing log ...
- 138
0
votes
1
answer
139
views
How to scale a subset of data with respect to the entire dataset
I am developing a financial time-series prediction model using sklearn using StandardScaler for scaling purposes. I train a model, and then use the model regularly ...
- 111
0
votes
2
answers
526
views
Should I split data into train/validation/test before feature scaling and feature selection or after?
I'm working on a project, I finished data preprocessing, and I found an article where it says that feature scaling and feature selection should be done after splitting data, some other articles say it ...
- 11
0
votes
1
answer
345
views
Feature scaling in Linear Regression
I always use Linearregression() class in sklearn library for creating a linear regression model. According to my understanding, we need feature scaling in linear ...
- 35
2
votes
1
answer
1k
views
Predict actual result after model trained with MinMaxScaler LinearRegression
I was doing the modeling on the House Pricing dataset. My target is to get the mse result and predict with the input variable
I have done the modeling, I'm doing the modeling with scaling the data ...
- 123
3
votes
1
answer
72
views
What justifies feature scaling?
Although I can understand the significance of feature scaling in some cases (e.g. when gradient descent is involved), I don't feel I understand the necessity of this process in general. But there a ...
- 181
2
votes
0
answers
27
views
When is scaling and centering important?
There are some models such as PCA or SVM where scaling and centering of training data is essential.
There are some models, mostly tree-based where scaling and centering is not required at all.
I don't ...
- 121
0
votes
1
answer
59
views
How to Scale target feature
How should I scale target feature? Should I use scaler as fit_transform on y_train, and just fit on y_test to avoid leaking data?
2
votes
1
answer
44
views
Choosing Right Optimiser and Data Scaling
The choice of optimiser and how data is scaled are both very important things in machine learning, yet they are not hyperparameters (as far as I am aware). It is also not necessarily obvious which ...
- 91
1
vote
2
answers
112
views
Ways of scaling scores on data without knowing possible maximum values
In my scenario, I have to process some input data and give a score based on what the processing phase outputs. The problem is that, in order to scale the score in a human-readable format I'd have to ...
1
vote
1
answer
281
views
Is test data required to be transformed by training data statistics?
I am using a dataset (from literature) to build an MLP and classify real-world samples (from wetlab experiment) using this MLP. The performance of MLP on the literature dataset are well enough. I am ...
0
votes
0
answers
105
views
Do I need to encode numerical variables like "year"?
I have a simple time-series dataset. it has a date-time feature column.
...
- 1
0
votes
1
answer
71
views
Why does log-transforming the target have a huge impact on MSE value?
I am doing linear regression using the Boston Housing data set, and the effect of applying $\log(y)$ has a huge impact on the MSE. Failing to do it gives MSE=34.94 ...
- 119
1
vote
1
answer
3k
views
Feature selection before or after scaling and splitting
Should feature scaling/standardization/normalization be done before or after feature selection, and before or after data splitting?
I am confused about the order in which the various pre-processing ...
- 119
0
votes
2
answers
80
views
Standardization in combination with scaling
Would it be ok to standardize all the features that exhibit normal distribution (with StandardScaler) and then re-scale all the features in the range 0-1 (with <...
- 119
2
votes
2
answers
235
views
Is it wise to always `StandardScaler()` features? [SOLVED]
My current investigations point to the sklearn.preprocessing.StandardScaler() not always being the right choice for certain types of feature extractions for neural ...
0
votes
1
answer
62
views
Scaled-input-trained ML Models
I have input data, the magnitudes vary quite a lot between features. I have scaled them using sklearn's StandardScaler(), then used keras to train a NN on this data to predict my target. I have ...
- 91
0
votes
1
answer
5k
views
__init__() takes 1 positional argument but 4 were given sklearn standard scaler error
I defined a class like below:
...
0
votes
1
answer
127
views
Feature scaling on null values
How to handle null values in dataset for performing feature scaling on a particular column?
i.e.Should we keep the null value as it is or impute some other value?
Is there any tutorial on how to ...
1
vote
0
answers
26
views
Feature Scaling + Selection when target is imbalanced
If my target is imbalanced, when should I do target balancing in preparation for modeling?
Before feature scaling and selection? After feature scaling and selection?
If I am doing backward elimination,...
- 11