Questions tagged [feature-scaling]

Feature scaling is a data pre-processing step where the range of variable values is standardized. Standardization of datasets is a common requirement for many machine learning algorithms. Popular feature scaling types include scaling the data to have zero mean and unit variance, and scaling the data between a given minimum and maximum value.

Filter by
Sorted by
Tagged with
0 votes
1 answer
39 views

Min-Max Scaling more sensitive to outliers than 'Simple Feature Scaling'?

I am confused as to the pros and cons of two different approaches to normalization: Min-Max Scaling, and what the lecturer in the course I am taking refers to as 'Simple Feature Scaling'. The latter ...
Chris Bedford's user avatar
2 votes
4 answers
79 views

How can I scale my data for a machine learning model in a way that preserves the relationship between columns?

Lets take a simple database with 3 columns called x1, x2 and label for example label is being labeled by this condition if x1-x2> 0 then label = 1 else 0 , i.e <...
Rushabh Kheni's user avatar
1 vote
0 answers
39 views

Scaling imbalanced binary features

I am interested in a discussion in encoding and scaling categorical features, notably imbalanced categorical features. The context is neural networks (gbdts should handle this easily). It is known ...
Lucas Morin's user avatar
  • 2,196
0 votes
0 answers
24 views

Little to no difference in linear regression plotted line

As I'm learning about basic machine learning concepts, I've learnt about linear regression. Part of my assignment was to implement a linear regression algorithm on a rather simple dataset, consisting ...
Aishgadol's user avatar
  • 111
1 vote
1 answer
48 views

How to normalize the features without the knowledge of the min and max values in online learning?

I am developing an online learning platform where input features are gathered from various sensors. However, these features may have vastly different ranges. For example, displacement values may be ...
JJbow's user avatar
  • 15
0 votes
0 answers
9 views

Deriving consistency feature for a student using a study app over the days

I want to build a recommendation engine for the revision app. Basic Idea After each module we will ask student questions and based on the correctness of their answers we will decide after how many ...
Keshav Raj's user avatar
1 vote
1 answer
31 views

Out-of-Range Target Variable in Sequence-based Machine Learning Model

I'm encountering a scaling issue in a machine learning project. I'm predicting a target variable from an input sequence (and doing this for many). However, I've encountered a challenge where the ...
Bloggy's user avatar
  • 11
0 votes
0 answers
15 views

How to deal with a small extra cluster in a tabular data?

I am working on a high dimensional tabular dataset with 1600 features and 9440 rows. No matter how I select the features, when I try to project my data into a 2d or 3d graph using dimensionality ...
Tanmay Sharma's user avatar
1 vote
1 answer
55 views

Scale dataset while preserving relative distributions between columns

I have a large dataset with 460 columns. The columns have names such as 'AppOpen_1day', 'AppOpen_2day', ...... 'AppOpen_15day', 'Dig_Pos_1day', 'Dig_Pos_2day', ...... 'DigPos_15day' etc. Each column ...
SacredMechanic's user avatar
0 votes
0 answers
130 views

K nearest neighbor with varying feature length

I'm trying to build a tool which predicts the elemental composition of some light source using its emission spectrum with the k nearest neighbor algorithm (I'm using the KNeighborsClassifier from ...
oodani's user avatar
  • 1
0 votes
0 answers
42 views

How to work with multiple feature types on autoencoder?

This is my first post here. I am working on an adversarial autoencoder that receives different features, encodes them, and decodes them. For instance, suppose you have a dataset from a large survey ...
Umberto Mignozzetti's user avatar
0 votes
0 answers
9 views

Which Frameworks/Libs Best Support Integer-Based Features, Scaling, Training, etc?

Papers such as Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference have interested me in exploring integer-based data science. In particular, I'm thinking of ...
ezekiel68's user avatar
0 votes
0 answers
15 views

Best distance metric and estadarization method for clustering with percentages data

I'm studying access patterns to a facility with clustering. My variables are percentages. For example, for each user, I have the percentage of access 'in time' versus late, or the percentage of using ...
Kaikus's user avatar
  • 101
1 vote
0 answers
19 views

Clustering Similar Articles Using Mixed Data: Seeking Advice and Validation

Question: I'm working on a project where I need to cluster a dataset of articles based on various features, including text, numeric values, and categorical data. I've implemented a clustering approach ...
sara sara's user avatar
0 votes
2 answers
44 views

Scaling of categorical feautres

In the context of algorithms that consider the scales of features, I have a situation where some features are encoded using ordinal encoding, some features are binary, and some features are standard ...
jxqbbb's user avatar
  • 51
0 votes
1 answer
34 views

Different scaling methods of different features results in a faux dependency between them

My dataset contains the following two features: "movie duration" (minutes) and "tv shows duration" (seasons). If a certain sample is of type "movie", it's duration will ...
Shir's user avatar
  • 1
0 votes
1 answer
13 views

feature engineering mechanism

why do we need to rescale some feature having large range I know we do it for faster rate of gradient descent ,but still how does rescaling works? and it doesn't break the model and does rescaling ...
rushi jhala's user avatar
1 vote
4 answers
130 views

Are scalers or encoders supposed to be serialized along with trained models?

Consider the very basic example below: ...
Muhammad Usman's user avatar
1 vote
1 answer
27 views

When training a sklearn machine learning model, what part of a data from a csv file needs scaling like MaxAbsScaler or MinMaxScaler?

Consider the code below: ...
Muhammad Usman's user avatar
0 votes
0 answers
7 views

Feature Selection using Statistical Testing on Features with Different Scales

If I want to select features based on ANOVA test for example, should I scale features to same/similar range so the results can be compared , or is it unnecessary?
jxqbbb's user avatar
  • 51
0 votes
1 answer
38 views

Standard Scaling not resulting in good nn inputs

I am using standard scaling to transform my dataset for input to a NN. The normalized dataset is in the range +-0.04. I am getting different results in the NN when simply multiplying by a factor of ...
BAR's user avatar
  • 101
5 votes
2 answers
899 views

How to handle similarity search on mixed data types vectors?

I think this question is one that many beginners run into and I could not find a decent generic guide for it. My issue is the following. I want to evaluate similarity of vectors which have mixed data ...
Chapo's user avatar
  • 53
1 vote
1 answer
141 views

How to encode & scale IP addresses as input for ML models

Im currently working on an anomaly detection while making a transaction. As a part of the data that I extracted, I have the IP addresses of the indivduals who made the transaction. Since the IP ...
Sivadithiyan official's user avatar
1 vote
1 answer
761 views

StandardScaler and MinMaxScaler vs RobustScaler

I've recently read that Standard Scaler functions best in situations where the distribution of the features are approximately normal. MinMaxScaler works in a way that it preserves the features' ...
Arthur Caldas's user avatar
0 votes
0 answers
41 views

Logistic Regression, Standardization, Stationarity, Differencing

I am going to be using the logistic regression in which I will use L2 Regularization. I have these 4 rolling standard deviation variables. Here are the results of the Augmented Dickey-Fuller Test for ...
DomIsAwesomee's user avatar
0 votes
1 answer
387 views

Combining text and image features with different scales

I have computed text features using [SBERT][1] and image features using VGG-16. The text features range from -1.58 to 1.58, whereas the image features range between 0 and 521. I would want to ...
Dan G's user avatar
  • 1
0 votes
1 answer
134 views

How to treat categorical columns after ordinal encoding?

If encode three categorical variables like "bad", "normal", "good" into 0,1,2, after that can I treat them as numerical values? So can I perform on them MinMaxScaler or ...
Flavio Brienza's user avatar
1 vote
2 answers
567 views

problem with standardScaler

problem with standardScaler hi I'd like to scale one column in the titanic data set. I am using the following code segment. for some reason df_scaled results an empty set. how can I solve it? what is ...
Mehmet Deniz's user avatar
1 vote
1 answer
72 views

Customer Segmentation with mixed data [closed]

I want to perform clustering. I am reading about this topic but I am totally confused. My dataset has 490 observations and it consists of numerical data (3 columns: Recency, Frequency, Monetary), ...
ArgyGr's user avatar
  • 11
2 votes
0 answers
304 views

Is there a need to use scaling for Age attribute?

What is the good way working with 'Age' attribute? Don't touch it or should it be scaled? Below photo shows my results 'Before' and 'After' standardization.
hyper-cookie's user avatar
0 votes
1 answer
39 views

Success metric of database migration using row counts

Description I have a problem where I'm tasked to successfully transform and repurpose data from one SQL server to another. Call the source $\text{src}$ and the target database $\text{tgt}$. In order ...
NoVariation's user avatar
1 vote
1 answer
127 views

Is it valid to use Spark's StandardScaler on sparse input?

While I know it's possible to use StandardScaler on a SparseVector column, I wonder now if this is a valid transformation. My reason is that the output (most likely) will not be sparse. For example, ...
user12138762's user avatar
1 vote
1 answer
324 views

Inverse Scaling Partitioned Data

I have scaled an original matrix A with sklearn's StandardScaler, resulting to a matrix S. I then partitioned the result into ...
ccccc's user avatar
  • 11
0 votes
2 answers
116 views

Understanding the relationship between two features

I am trying to solve classification task. Could you suggest me, if these two features are independent? the plot looks strange
Любовь Пономарева's user avatar
0 votes
1 answer
1k views

Scaling the target variables for Neural Networks in Regression Problem

I am trying to implement a neural network on a regression problem. I have scaled the independent variables since this is a crucial step for neural networks. However, I see online that some people ...
John adams's user avatar
0 votes
1 answer
424 views

Standardization after log transformation

I have a few question about log transformation and standardization. First: Should I standardize my features after doing log transformation? Second: I still do not understand, because when doing log ...
Jovian Aditya's user avatar
0 votes
1 answer
214 views

How to scale a subset of data with respect to the entire dataset

I am developing a financial time-series prediction model using sklearn using StandardScaler for scaling purposes. I train a model, and then use the model regularly ...
functorial's user avatar
0 votes
2 answers
1k views

Should I split data into train/validation/test before feature scaling and feature selection or after?

I'm working on a project, I finished data preprocessing, and I found an article where it says that feature scaling and feature selection should be done after splitting data, some other articles say it ...
biihu's user avatar
  • 11
0 votes
1 answer
550 views

Feature scaling in Linear Regression

I always use Linearregression() class in sklearn library for creating a linear regression model. According to my understanding, we need feature scaling in linear ...
AAA's user avatar
  • 35
2 votes
1 answer
2k views

Predict actual result after model trained with MinMaxScaler LinearRegression

I was doing the modeling on the House Pricing dataset. My target is to get the mse result and predict with the input variable I have done the modeling, I'm doing the modeling with scaling the data ...
MADFROST's user avatar
  • 123
3 votes
1 answer
117 views

What justifies feature scaling?

Although I can understand the significance of feature scaling in some cases (e.g. when gradient descent is involved), I don't feel I understand the necessity of this process in general. But there a ...
ado sar's user avatar
  • 181
2 votes
0 answers
42 views

When is scaling and centering important?

There are some models such as PCA or SVM where scaling and centering of training data is essential. There are some models, mostly tree-based where scaling and centering is not required at all. I don't ...
xan's user avatar
  • 121
0 votes
1 answer
68 views

How to Scale target feature

How should I scale target feature? Should I use scaler as fit_transform on y_train, and just fit on y_test to avoid leaking data?
Rus Zzzeta's user avatar
2 votes
1 answer
61 views

Choosing Right Optimiser and Data Scaling

The choice of optimiser and how data is scaled are both very important things in machine learning, yet they are not hyperparameters (as far as I am aware). It is also not necessarily obvious which ...
Socorro's user avatar
  • 101
1 vote
2 answers
183 views

Ways of scaling scores on data without knowing possible maximum values

In my scenario, I have to process some input data and give a score based on what the processing phase outputs. The problem is that, in order to scale the score in a human-readable format I'd have to ...
Ionut-Alexandru Baltariu's user avatar
1 vote
1 answer
530 views

Is test data required to be transformed by training data statistics?

I am using a dataset (from literature) to build an MLP and classify real-world samples (from wetlab experiment) using this MLP. The performance of MLP on the literature dataset are well enough. I am ...
Apollonia Vitelli's user avatar
0 votes
0 answers
159 views

Do I need to encode numerical variables like "year"?

I have a simple time-series dataset. it has a date-time feature column. ...
smarks70's user avatar
0 votes
1 answer
143 views

Why does log-transforming the target have a huge impact on MSE value?

I am doing linear regression using the Boston Housing data set, and the effect of applying $\log(y)$ has a huge impact on the MSE. Failing to do it gives MSE=34.94 ...
Caterina's user avatar
  • 119
1 vote
1 answer
4k views

Feature selection before or after scaling and splitting

Should feature scaling/standardization/normalization be done before or after feature selection, and before or after data splitting? I am confused about the order in which the various pre-processing ...
Caterina's user avatar
  • 119
0 votes
2 answers
147 views

Standardization in combination with scaling

Would it be ok to standardize all the features that exhibit normal distribution (with StandardScaler) and then re-scale all the features in the range 0-1 (with <...
Caterina's user avatar
  • 119

1
2 3 4 5 6