I'm currently trying to create a neural network for playing Tetris. I'm using evolutionary algorythms for it's learning, so the behavior that I need to get from the neural network is the following:
- Create a RNN model
- Give it a single input (current field state) and take an answer that takes into account previous moves
- Use the neural network response to make a move in the game
- Get new game field state, pass it to the neural network, repeat from point (2)
The problem is that on the same inputs I get the same outputs, which indicates that the RNN does not take into account its previous actions:
model = keras.Sequential(
[
layers.SimpleRNN(10, return_sequences=True) ,
layers.SimpleRNN(10, return_sequences=True) ,
layers.Dense(3, input_dim =10, activation='softmax')
]
)
x1 = np.random.rand(1, 1, inp_size)
y1 = model(x1)
y2 = model(x1)
y3 = model(x1)
print(y1)
print(y2)
print(y3)
Output:
tf.Tensor([[[0.3274583 0.42117843 0.25136325]]], shape=(1, 1, 3), dtype=float32)
tf.Tensor([[[0.3274583 0.42117843 0.25136325]]], shape=(1, 1, 3), dtype=float32)
tf.Tensor([[[0.3274583 0.42117843 0.25136325]]], shape=(1, 1, 3), dtype=float32)
So, when i do the same thing, but feed input as a sequence packed to tensor:
x = np.ones((1, 3, inp_size))
y = model(x)
print(y)
I get different results on the same inputs:
tf.Tensor(
[[[0.23801968 0.4496906 0.31228966]
[0.30908147 0.3184522 0.37246636]
[0.2849004 0.5014079 0.21369171]]], shape=(1, 3, 3), dtype=float32)
How can I create a RNN model which will not need a full match history as an input sequence to take proper decisions? The question is about opportunity to force network to work step by step, as if processing the sequence, but at the same time receiving only a SINGLE element at the input and immediately giving an answer (since the n-th element of the input sequence depends on the n-1 network output, and we cannot get it until previous output will not be processed by the game)