My data set is of shape (1249, 228). Most of the entries are zero and other are integers like 1,2,5,10,20 etc. I want to transform this set for the input into LSTM. But when I am applying MinMaxScaler. It is giving the following error:

load the dataset:

dataset1 = pd.read_csv('g:/hello.csv', engine='python')
dataset1= dataset1.drop('packages', axis=1)
dataset1 = dataset1.astype('float32')

normalizing the set

scaler = MinMaxScaler(feature_range=(0, 1))
dataset1 = scaler.fit_transform(dataset1)

ValueError: Input contains NaN, infinity or a value too large for dtype('float64').

How can I transform this data set according to the input in LSTM.


I assume you checked for NaN and Inf values manually.

According to the solutions posted here: https://stackoverflow.com/a/44869902/6204860 and https://datascience.stackexchange.com/a/11933/52089

if dataset1 is a Pandas DataFrame try converting it to a matrix by running this:

dataset1 = dataset1.as_matrix().astype(np.float)

and then run dataset1 = scaler.fit_transform(dataset1).

Let me know if this works :)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.