Questions tagged [normalization]

The tag has no usage guidance, but it has a tag wiki.

Filter by
Sorted by
Tagged with
-1
votes
0answers
15 views

Effect of label normalization on optimization?

Let's say in a regression task I have a range of labels 1-60. If I normalize the labels and squeeze those into 0-1 range (by dividing 60) and calculate loss then the calculated loss will be very small ...
0
votes
1answer
27 views

What is an example of normalization (making the row unit norm)?

I am learning about standardization and normalization concepts for feature engineering. Standardization is done for example using z-score where based on the mean and std deviation we re-calculate the ...
0
votes
1answer
34 views

How to normalize noisy data

Suppose I have 1-D data which has some outliers, I want to normalize the data to be in the range [0,1]. I tried calculating the maximum value and the minimum value as follows: ...
2
votes
2answers
93 views

Should normalization be applied?

I have more then 100 columns with the values of 1-0. But the two features at the end as seen in the below image, have different values then the rest. Should I rescale the values in the last two ...
0
votes
1answer
25 views

How to use a multiple linear regression model built from normalized data

I built a linear multivariable regression model from normalized data (for the interval [0; 1]). Initially, the data was not normalized, I normalized the data by myself (independent and dependent ...
1
vote
0answers
21 views

Normalisation of features extracted from audio files

I am building CNN and SVM models which take in MFCC features as input. The MFCC matrices shape is (13, n). The 13 rows are coefficients and n columns represent n time frames. So each row in the matrix ...
0
votes
1answer
31 views

Normalize data with extreme outliers for forecasting

Suppose I have input values that represent the change of a stock share from each time step to the next. Now I want to feed these values into an LSTM Neural Net. My problem is that most values are ...
0
votes
2answers
20 views

Selecting Transforms with sklearn pipelines

So I am currently working on a Data set, and I want to use Pipelines to select the transforms. Here is an example of what I want to do : ...
1
vote
1answer
25 views

Logistic Regression from scratch in numpy - Is data normalization needed?

I was trying to implement Logistic Regression from scratch in python to learn better how it works under the hood. In particular I am following this video tutorial from Andrew Ng. This is the dataset I ...
0
votes
0answers
17 views

Normalizing data of different observation points

I want to normalize my data, but I am too stuck with the specifics of it. I have observations in weekly intervals from different groundwater wells, and as expected they have a different range of ...
0
votes
0answers
29 views

Performance worsen when normalizing on CNN

I am facing quite a strange behaviour. As far as I have understood, especially when dealing with CNN, feature normalisation/ standardisation, should help the model at converging faster. Now, I am ...
0
votes
0answers
22 views

Neural network output is 0 for test data (using RELU for activation)

Maybe this is a naive question, but I have a NN that uses relu for all layers. In train data there is no problem, but in test (or validation) the outputs are all 0. ...
0
votes
0answers
10 views

Does it make sense to train kalman filter on unscaled (Timeseries-)data, to clean it?

I have Timeseries data to clean and I got the tip to use a kalman Filter. My Question is, does it make sense to "fit" this kalman-filters parameters to an unscaled and unnormalized (...
2
votes
2answers
47 views

What is the best way to normalize a set of datasets

I have a data set that contains the same Time series "Sensor readings" for different days and I want to make a deep learning model to predict these values. What I did was I splatted the data ...
4
votes
1answer
118 views

feature scaling xgbRegressor

I read for example in this answer: Does the performance of GBM methods profit from feature scaling? that scaling doesn´t affect the performance of any tree-based method, not for lightgbm,xgboost,...
1
vote
1answer
76 views

Generating the right target for an LSTM model

Trying to explain my question on a simplified data set. Having the following dataset: ...
0
votes
1answer
33 views

Use both differencing and normalization in time series modeling to make it stationary?

I am working on a time series dataset. Should we use both differencing and normalizing or either of the ones to make it stationary?
0
votes
0answers
15 views

Understanding Scaling With Multiple Datatypes in a Neural Network

I've looked around and seen a huge amount of discussion on scaling inputs and targets for neural networks, but can't seem to find universal agreement on a few issues. Suppose I had a dataset, with ...
0
votes
0answers
11 views

Normalization around the nominal value while avoiding to get too big results

Lets say we measured a signal N times. Each measurement contains T time instance. For example for N=4 measurements and T=5 time instances the data matrix is as follows: ...
0
votes
0answers
20 views

Orange 3.25 Normalize features Center by Mean and Scale by SD

I am using Orange 3.25 on windows 10, But while I DOWNLOAD time series add on preprocessors I drag Normalize feaures, I dont see Center by mean and scale by SD Please help me how to get that .
1
vote
1answer
98 views

What is the procedure to realize zero mean and unit variance?

The tag on feature scaling says: Popular feature scaling types include scaling the data to have zero mean and unit variance, and scaling the data between a given minimum and maximum value. Are ...
0
votes
1answer
85 views

What is difference between Standard Normal Distribution and Mean Normalization approaches to feature-scaling?

The tag feature-scaling seems to convey that one of the scaling methods is Standard Normal Distribution. Further, I read an Answer on this site saying that Mean Normalization is a form of feature ...
1
vote
0answers
12 views

When should I reverse normalizations to evaluate loss?

If I am training a neural network and have normalized the data before-hand, should I reverse the normalization to calculate the loss? This tutorial provides an example of this method. What if I'm ...
1
vote
0answers
19 views

How to handle negative delays when predicting flight delays?

I am working with the nycflights dataset. My goal is to predict departure and arrival time delays using random forests. My ...
0
votes
0answers
30 views

How to normalize price from different years for machine learning models

I'm working on a price prediction problem which there is monthly data available with prices for several years. For training complex non-linear models like neural networks I will need a lot of data to ...
2
votes
1answer
153 views

Tensor Flow Time Series Tutorial Question

In the tutorial, they normalize the data and say "The mean and standard deviation should only be computed using the training data" What does this refer to? Why should you only use the training data?
0
votes
0answers
21 views

how do I approach forecasting problems using deep neural networks?

I am new to machine learning in general, and I have been requested to predict a price given a date. I have been trying to make a neural network for the task but it does poorly in the testing set, so I ...
1
vote
0answers
14 views

Normalizing historical data in time-series LSTMs

I am currently trying to solve a sequence prediction problem using LSTMs in a keras architecture. To illustrate the problem I give the following example which resemble the problem I must solve. Lets ...
0
votes
0answers
11 views

why does Scikit Learn's Power Transform always transform the data to zero standard deviation?

all of my input features are positive. Whenever I tried to apply PowerTransformer with box-cox method, the lambdas are s.t. the transformed values have zero variance. i.e. the features become ...
0
votes
1answer
24 views

Normalize Test and Control, or as a whole?

I have the following data: \begin{matrix} &Test Sample\ 1 & Test Sample\ 2 \dots & Test Sample \ 6& Control Sample\ 1 & Control Sample\ 2 \dots & Test Sample \ 6\\ ...
1
vote
1answer
17 views

Subdivide a numerical vector with a normal distribution

I have a numerical array of prices values. I'd like to do classification on this parameter, so I'd like to create a certain number of classes with the same granularity. I'd like to create a ...
0
votes
1answer
225 views

Trying to understand the result provided by np.linalg.norm function in numpy (normalisation)

I'm new to data science with a moderate math background. I'm playing around with numpy and can across the following: So after reading np.linalg.norm, to my ...
0
votes
1answer
100 views

(De-)Scaling/normalizing input and output data inside Keras model as layer

I am building a 2-hidden layer MLP using Keras. I'm using a SciKit learn wrapper to be able to use the GridSearchCV functionality. My sample-size is limited, ...
1
vote
1answer
48 views

When is z-normalization not needed when using DTW?

I'm hoping to get some answers to a question I have regarding normalization of DTW datasets, in particular datasets in which two time-series shapes with similar shapes but differences in magnitude are ...
1
vote
1answer
16 views

Should output data scaling correspond to the activation function's output?

I am building an LSTM with keras which have an activation parameter in the layer. I have read that scaling on the output data ...
1
vote
0answers
22 views

Should I normalize the target data the same way as the input data?

This is the way I'm currently transforming my input data ...
0
votes
0answers
17 views

Normalising time-series by geo location

I trained the LSTM model to forecast the number of people infected with COVID-19. Since each country and state (geo location) has different number of population as well as the infected over time, I ...
0
votes
1answer
24 views

Normalize data from different groups

I have data that have been grouped in 27 groups by different criteria. The reason for these groupings is to show that each group has a different behavior. However, I would like to normalize everything ...
2
votes
0answers
91 views

K-Means Clustering Profile Plot & Data Normalization

I am new to k-means clustering and I am working on a project on cryptoanalysis. I have a few questions and I hope to get some help here. I have four variables and my variables data values can range ...
0
votes
0answers
74 views

Custom layer in tensorflow which remembers input statistics

I would like to implement a layer which in a sense remembers what it's usual input is, and only passes on values that deviates from that. So basically, the layers need to remember the average of all ...
1
vote
1answer
42 views

Noramlization Time Series to Predict Stocks exact Price

I am trying to made a model for neural network that tries to predict prices of the stocks point by point using a LSTM (yeah I know that probably did not get anything and I should predict up/down or ...
0
votes
0answers
18 views

How do I apply Min Max scaling for numerical forecast when both dependent and independent volumes are increasing over time?

I'm want to build a numerical regression to forecast. From my initial analysis, it shows linear models (glm) out performs the typical decision tree models (xgboost, ranger...etc). I hypothesized that ...
0
votes
0answers
17 views

How do I discern document structure from differently-tagged XML documents?

I have a body of PDF documents of differing vintage. Our group had exported the documents as text to feed them into a natural-language parser (I think) to pull out subject-verb-predicate triples. ...
0
votes
0answers
5 views

The need to normalize input data for DNN when using He_initialization

I'm reading Geron's "Hands-On Machine Learning with Scikit_learn, Keras & Tensorflow" I noticed in the problem set after chapter 11, he ran a DNN using ELU activation and He Initialization. But ...
0
votes
0answers
19 views

When normalizing unstructured data, can you take the mean and standard deviation globally?

I'm watching this FastAI machine learning lesson, which is about the famous MNIST dataset. I've gottent to the point where we should normalize the data and, much to my surprise, the instructor uses ...
5
votes
3answers
379 views

How to normalize data without knowing the min and max values?

I have a Lending club dataset from Kaggle; it contains many different columns: there are for example dummy variables, years, amount of the loan...ect I want to normalize the data in the training and ...
1
vote
0answers
19 views

How Google Trends is normalized?

I have a daily series from Google Trends, using the range "today 3-m", but it comes that the last day is not available from this query. For example, today is March,24th and the last day using this ...
0
votes
0answers
38 views

How to correctly manage predictions when the inputs are outbound the original scaling range?

I have a neural network for a regression problem that was trained using MinMaxScaler(0,1) for features and I have two questions with this. I often find that scaling the output (or target variable) ...
0
votes
0answers
24 views

MAPE over 100% after normalization of dataset

I try to forecast power demand for next 24 hours. Years 2017 and 2018 are my training set, 2019 is test set. I use multistep vanilla LSTM . In first step I used original data with any preparation and ...
0
votes
1answer
16 views

Why normalize function has a different result on a matrix vs single value?

I have a matrix like: B=[ 1.5035; 1.5728; 1.6485; 1.5369; 1.5467; 1.572; 1.5374; 1.787; 1.5825; 1.6905]; Using normalize function like normalize(B,'range') ...

1
2 3 4 5