normalizing data and avoiding dividing by zero

Question

I have data that I'm compressing with AutoEncoders (3-layer neural network) and I would like to normalize my data first. I would like to try to use the coded latent vector and feed it into an anomaly detection algorithm and see what happens.

I would like to normalize the data for the autoencoder so my values are either between 0,1 or -1,-1 because my output activation function will either be a sigmoid or tanh. This way my algorithm can train and the input will be in the same range as the output values of the NN.

However, when I normalized with

x(i)-xmean/(xmax-xmin) 

I ended up dividing by 0 in several features of the data which gave NaN. Is is possible to normalize my data so it is between -1,1 or 0,1 while avoiding dividing by 0 for my data?

Answer 1

While you could do this manually, Python also has a handy little function called MinMaxScaler, which will automatically apply max-min normalization to scale data between 0 and 1.

Assume we have an array of 200 values for variables s and t:

import numpy as np
import pandas as pd
from sklearn.preprocessing import MinMaxScaler

mu, sigma = 20, 10 # mean and standard deviation
s = np.random.normal(mu, sigma, 200)
t = np.random.normal(mu, sigma, 200)

Reshape your variables if necessary:

s=np.reshape(s,(-1,1))
t=np.reshape(t,(-1,1))

Now, you can see that we are forming two new variables, snew and tnew, which we are scaling using MinMaxScaler.

scaler = MinMaxScaler()
print(scaler.fit(s))
print(scaler.fit(s))
snew=scaler.transform(s)
tnew=scaler.transform(t)

Here is a sample of our new variables:

>>> snew
array([[0.24896606],
       [0.63121206],
       [0.60448469],
       .......
       [0.49044733],
       [0.28131596],
       [0.32909155]

>>> tnew
array([[0.91224005],
       [0.74540598],
       [0.3938718 ],
       .......
       [0.75749275],
       [0.80709325],
       [0.19440844]

Answer 2

As others pointed out, you can normalize or standardize your data using the following steps. I'm sure other libraries have similar functions but I think this is efficient.

Since you requested normalization, I'll cover that topic in this post. As others alluded, data normalization is the process in which researchers or data science practitioners make all the values in a given dataset be proportionally spread between 0 and 1.

To implement normalization, follow the steps below:

from sklearn.datasets import load_iris
from sklearn import preprocessing

iris = load_iris()
print(iris.data.shape)

X_data = iris.data
y_labels = iris.target

normalized_X_data = preprocessing.normalize(X_data)