Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [feature-scaling]

Feature scaling is a data pre-processing step where the range of variable values is standardized. Standardization of datasets is a common requirement for many machine learning algorithms. Popular feature scaling types include scaling the data to have zero mean and unit variance, and scaling the data ...

1
vote
1answer
17 views

Is it better to use a MinMax or a Log Return normalization to predict stock price movements?

I am trying to use a LSTM model to predict d+2 and d+3 closing prices. I am not sure whether I should normalize the data with a MixMax scaler (-1,+1) using the log return (P(n)-P(0))/P(0) for each ...
0
votes
1answer
25 views

How to normalize data of a different nature?

I am working a price prediction LTSM model for the stock market. I am using multiple features: Open, Close, High and I would like to add the Volume. The 3 first features are of the same nature but ...
0
votes
2answers
24 views

Scaling features in artificial neural networks

So it is a well known thing that it is a good idea to scale features/training samples in the training set, so that the values do not differ too much in the absolute sense. For example we want to train ...
2
votes
1answer
25 views

Does Orange scale the data automatically for the linear regression with Ridge regularization

I'm using the linear regression tool with the Ridge regularization. To use the Ridge regularization I have to scale the data first. Does Orange scale the data automatically? I can't find any ...
1
vote
0answers
17 views

How standardizing and/or log transformation affect prediction result in machine learning models

I recently ran an elastic net model on my data. My predictors are mostly skewed. I found my model perform slightly better when I standardize on log-transformed data than standardizing on original data....
1
vote
0answers
14 views

Normalize data with uneven groups?

I have a dataset with 3 independent variables [city, industry, amount] and wish to normalize the amount. But I wish to do it with respect to industry and city. Simply grouping by the city and industry ...
0
votes
1answer
60 views

MinMaxScaler returned values greater than one

Basically I was looking for a normalization function part of sklearn, which is useful later for logistic regression. Since I have negative values, I chose MinMaxScaler with like so: ...
7
votes
1answer
179 views

What are some situations when normalizing input data to zero mean, unit variance is not appropriate or not beneficial?

I have seen normalization of input data to zero mean, unit variance many times in machine learning. Is this a good practice to be done all the time or are there times when it is not appropriate or not ...
1
vote
1answer
18 views

Using historical label as a feature in my ML model?

I am working on a predictive model to predict change in the price of an asset (up, down, no change). The labeling is based on the derivative of the price and is exponentially smoothed with an alpha of ...
2
votes
1answer
16 views

The use of feature scaling in scikit learn

I'm studing machine learning from here and the course uses 'Scikit Learn' for regression - https://www.udemy.com/machinelearning/ I can see that for some training regression algorithms, the author ...
1
vote
1answer
18 views

How to deal with Optional Input

I'm from the vision world and only worked with pixels from 0-255, ignoring any side effects. My current problem is different, in the way that I cannot rely on the input data. What my problem is: I ...
0
votes
1answer
27 views

Input standartization for Deep Learning - Proper Scaling

Typically the input to neural network (NN) is transformed to have zero mean and 1 std. I wonder why std scale should be 1? What about other scales? 10? 100? Doesn't it make sense to provide NN with ...
0
votes
0answers
11 views

Does it make sense to preprocess (normalise or standardise) this data for GAN?

I'm working on a project where I have a dataset for a dynamical system (pendulum) containing a trajectory, energy cost and corresponding control actions (See below). I'm using a generative adversarial ...
0
votes
0answers
7 views

Handling Error for Continuous Features in a Content-Based Filtering Recommender System

I've got a content-based recommender that works... fine. I was fairly certain it was the right approach to take for this problem (matching established "users" with "items" that are virtually always ...
0
votes
0answers
38 views

Noise Resulting from Inverse-Scaling for a Machine Learning Problem

I have created a neural network and am predicting reasonable values for most of my data (the task is multi-variate time series forecasting). I scale my data before inputting it using scikit-learn's ...
3
votes
1answer
494 views

Data scaling before or after PCA

I have seen senior data scientists doing data scaling either before or after applying PCA. What is more right to do and why?
1
vote
2answers
27 views

How to standarize feature vector with data in different scales?

Let's suppose I have a dataset with numerical attributes of different types. Let's suppose I want to employ a Neural Network for supervised classification with that dataset. For that, I need to ...
1
vote
3answers
87 views

How to Normalize a feature [closed]

I have a feature that income of individual. It ranges from 10k to 116 Million. I have about 300k+ records. Clearly, I cannot use this feature as is as it will distort the model output and there are ...
1
vote
0answers
9 views

how to model the implication of a feature in classification

I have a binary classification problem in which one feature (let's call it X) has implication property, i.e. if the feature is 1, the output of classification must ...
1
vote
0answers
20 views

Should I rescale tfidf features?

I have a dataset which contains both text and numeric features. I have encoded the text ones using the TfidfVectorizer from sklearn. I would now like to apply logistic regression to the resulting ...
0
votes
0answers
28 views

scaling features after imputing missing values

I have a set of features with missing values. After imputing them by the median, I perform feature scaling (in Python/scikit i use preprocessing.scale and imputer). Now, there are many zeros which ...
1
vote
0answers
8 views

Choosing parameters for testing

I have a DL4J project DL4J Framework for a CNN for Classification and Scaling. (Separate projects). My question is that how can I tune my parameters to achieve my goal? Note that my question is ...
0
votes
0answers
19 views

Scatter Plot on Map within a Small Range (a university campus)

I am trying to get a scatter plot of points based on geolocation data superposed on a map. I have already went through several libraries like basemap or plotly which do similar stuffs. However, the ...
2
votes
1answer
260 views

Data scaling before PCA: how to deal with categorical values?

I have to apply PCA on a dataset, which contains both numerical and categorical values. In the preprocessing phase, I converted all the categorical values in numerical, so that the software can deal ...
2
votes
1answer
192 views

How to normalize a boolean feature for neural nets?

I have a feature that is boolean and I would like to feed it to a neural net as one of the inputs. I think in theory the best is to encode as false->0 and true->1 because 0 as an input will deactivate ...
9
votes
3answers
2k views

Zero Mean and Unit Variance

I'm studying Data Scaling, and in particular the Standardization method. I've understood the math behind it, but it's not clear to me why it's important to give the features zero mean and unit ...
3
votes
1answer
262 views

Should one hot vectors be scaled with numerical attributes

In the case of having a combination of categorical and numerical Attributes, I usually convert the categorical attributes to one hot vectors. My question is do I leave those vectors as is and scale ...
2
votes
0answers
22 views

How to weigh feature array

I have a feature array of around 4000 elements, extracted from one source. On this array I've extracted 7 more feature from other source and now I basically have a 4007 feature array from each data ...
6
votes
4answers
85 views

Prediction from data series with varying features

I'm looking into a problem where the data points have unequal features. Each instance represents the progression of an item throughout the system. A number of them have progressed to their end point,...
2
votes
1answer
195 views

Using z-score for neural network normalization

I've read many people use z-score to normalize their data for presenting to a neural net, and that all data should lie in a range (usually -1 to 1), but z-score can return results beyond those bounds. ...
1
vote
1answer
192 views

How to Normalize & Scale a Single Data Point

I do understand the concept of normalizing & scaling the training/test data; it does help with the converging of the cost function. It is a great helper for many of the machine learning algorithms....
0
votes
1answer
2k views

How to use the same minmaxscaler used on the training data with new data?

Im using the keras LSTM model to make prediction, and the code above is to scale the data: inputs are shaped like (n, 11, 1) and the label is 1D DailyDemand.py ...
0
votes
1answer
51 views

How to do Feature Scaling for these ranges [0,1] and [-1,1]?

I want to rescale the features of my data to be between [0,1] and [-1,1]? Is their a clear cut way that works every time for these ranges? I think the below equation works for [0,1] but when it is ...
1
vote
1answer
44 views

standardize dataset with numerical and dummy features

I have a dataset with both numerical and categorical features (variables), I converted all the categorical variables in dummies, then I split the train and test data. Now I am at the step where I ...
10
votes
2answers
181 views

Linear Regression and scaling of data

Following plot shows coefficients obtained with linear regression (with mpg as target variable and all others as predictors) for mtcars dataset (see https://stat....
0
votes
1answer
357 views

Scaling multiple time series data

I am using crypto currency chart data and was wondering what would be the best process for scaling the time series data. The issue is that some of these currencies are highly volatile and can see ...
0
votes
0answers
122 views

Normalization when using tanh function in RNN

I am training a RNN model using GRUCell in Tensorflow. I tried two approaches, first using default tanh activation function for GRUCell and second using leaky_relu (...
0
votes
0answers
54 views

Should I do feature scaling if the data I have is a series of 20 * 20 matrix

I'm here again hoping to get some answer to my question. I have hundreds of series audio and on each audio I pulled MFCC feature using this function ...
1
vote
2answers
490 views

Different number of features after using OneHotEncoder

I have train and test data in two separate files. OneHotEncoder gives different number of features for Train and Test Data based on the different values they have. But the classifier requires that ...
2
votes
0answers
32 views

What are best practices for collaborative feature engineering?

I work in a large company on several data science projects. For each of the projects me and my colleagues construct features that have some predictive value for the specific target in that project. ...
2
votes
1answer
452 views

Simple neural network implementation in keras [closed]

I have a binary classification problem(benign/malicious) and I have applied simple neural network with one hidden layer for solving the problem. I have 46 features in my dataset and for the hidden ...
3
votes
2answers
114 views

Mean and Variance of Feature Scaling

Many people use the mean and variance of the training set to standardize the test set, instead of calculating the mean and variance of the test set and use these. Isn't risky to do that ? If no, why ?
1
vote
1answer
114 views

Clustering combining numeric features and weekday & hour cyclic features

The question is strictly related to What is a good way to transform Cyclic Ordinal attributes? and Ways to deal with longitude/latitude feature They presented a very clear answer about the approach ...
2
votes
0answers
337 views

How to scale data for LSTM autoencoder?

I am working on an LSTM autoencoder in keras. The aim here is to obtain a latent space representation for the time sequences which I intend to use for clustering. My input sequences (each feature) ...
-1
votes
1answer
60 views

How to persist data scaler for predictions

I have a Support Vector Machine in Scikit-learn (Python) that gets trained once in a while when enough new data has accumulated (user help train the model by submitting new data). I store the model ...
0
votes
1answer
250 views

Features standardization - Multilayer perceptron

I have serious doubts concerning the features standardization done before the learning process of a multilayer perceptron. I'm using python-3 and the scikit-learn package for the learning process and ...
-2
votes
1answer
36 views

How to redistribute weightage proportionally?

I want to increase proportional increase weightage. For example, I have weights w1 = 0.4 w2 = 0.3 w3 = 0.2 w4 = 0.1 with a constraint that the total sum of the ...
1
vote
0answers
538 views

Preparing, Scaling and Selecting from a combination of numerical and categorical features [closed]

I'm currently working on the Titanic dataset from Kaggle. The features consist of both numerical and categorical variables and I've also engineered a few categorical variables using original features. ...
1
vote
1answer
3k views

How to use the same scale with new data? - scikit learn - scikit learn

How do I use the same scale used in preprocessing with new data. Actual code: ...
0
votes
1answer
2k views

Best way to normalize datasets for a linear regression model?

I am having some trouble getting a proper fit for a line using a simple linear regression model in tensorflow. I've taken some housing data which I've normalized using the standard algorithm (...