I have 400 time series of length 50. 200 of them have values between 1-10 and are considered of type A. The rest 200 have values 1-10 with the exception that 3 from the total of 50 data points have value 20, and are considered of type B.
I am trying to use a RNN network to make it learn these differences using tensorflow.
I use GRU cell with input size is 50 (the whole time series), internal state is 100, 3 layers, feature length is 1 and dropout 0.8. Batch size is 1 (1 whole time series of 50 values) 80% of the series are used for training and 20% for test, evenly distributed between type A and type B.
I use softmax as activation function and for gradient descent optimization i use RMSPropOptimizer. Also i am not changing the data at all before feeding it into the neural network, i tried normalizing them using (x-min()/max()-min()) but it didn't work.
When i am trying to train it, i always get 50% error. Maximum epochs i tried are 10 and it still was 50%.
The same network can classify correctly if i use it on data that have way more differences than just on these 3 points, even on small dataset of about 50 runs of each type.
Is it impossible for a RNN to learn these small differences on just those 3 data points out of the 50? What would you suggest i can change in my configuration? I also tried with 2000 time series and still the error is the same.
Also note that these 3 data points exist at the beginning of the time series. (in case this has something to do with my issue).
