Time Series Classification using LSTM

Question

I have multiple .csv's files, each of them represent a product. I am using LSTM to classify these products as good or bad. All .csv's have been clubbed together in form of a 3d matrix of (#files, time_steps, #features). I am passing the matrix in batch_size=128.

````below is the psuedo code. model = Sequential() model.add(LSTM(#cells(64), input_shape=(time_steps, #features))) model.add(Dense(1, activation='sigmoid')) model.compile(loss='binary_crossentropy', optimiser='adam', metrics=['accuracy']) model.fit(train_X, train_y, batch_size=128, epochs=100, validation_split=0.1)

The problem is that I am getting NaN's in loss function and accuracy too is 0.000.

Can anyone suggest me where to look. Thanks in advance.

Answer 1

There might be many reasons for NAN's-

https://stats.stackexchange.com/questions/325451/cost-function-turning-into-nan-after-a-certain-number-of-iterations

https://stackoverflow.com/questions/33962226/common-causes-of-nans-during-training

basically LSTM is not able to train. try changing parameters esp activation function.

I haven't worked on python but what is the use of metrics=['accuracy']) in ur code, when u have already given cross entropy as loss function.

Answer 2

First thing i suggest you to try is to change the learning rate. Change the following code

model.compile(loss='binary_crossentropy', optimiser='adam',metrics='accuracy'])

to

from keras import optimizers
adam = optimizers.Adam(lr=1e-5)
model.compile(loss='binary_crossentropy', optimiser=adam,metrics='accuracy'])

Keep reducing Learning rate and check if the same problem is happening.