# Recurrent neural network producing same predictions

I am trying to train a recurrent neural network that I built in keras on timeseries data to predict number of sales for next 10 days. For this, I've created my dataset as -

var(t) -> var(t+1)
var(t+1) -> var(t+2)
var(t+2) -> var(t+3)
var(t+3) -> var(t+4) and so on


I did Min-Max scaling on this data and the RNN code is as follows -

model = Sequential()
model.compile(loss='mean_squared_error',optimizer=opt)
model.fit(xtrain, ytrain, epochs=100, batch_size=1, verbose=2)


But the plot I am getting is when I did predictions on xtrain (green = ytrain, blue = ypred) -

The rnn isn't learning anything at all. Its producing same results for each dataset. I've tried adding hidden layers, increasing number of neurons, changing parameters (learning rate, momentum), optimizers (sgd, adam, adagrad, rmsprop), lstm activation fxn (tanh, softsign). I got little fluctuations in some cases in the graph. But the output is mostly constant. Also, I've only 200 datasets.

Can someone please guide me what I am doing wrong here. What else I can try. Will small sized data not work using RNN at all ? If so, is there any other way to solve this problem (except ARIMA model) ?

EDIT - Increased batch size to 100 and epochs to 1000. Received some better results. Also, I did mean normalization [(x-mean)/std_dev] instead of MinMax scaling.

• input shape is suspicious. what value as look_back? did you import Adam optimizer as such keras.optimizers.Adam as adam? learning rate of 0.1 is quite high. batch_size of 1 looks small. – tagoma Jul 14 '17 at 20:04
• @edouard look_back is 1. yes, adam optimizer is from keras.optimizers.adam I've tried learning rate values - 0.01, 0.001, 0.03, 0.3 - none worked. – Ishan Jul 14 '17 at 20:12
• @edouard I've edited the question with some changes that you mentioned. Decreasing learning rate to 0.01 didn't help. – Ishan Jul 14 '17 at 20:27