0
$\begingroup$

I'm currently trying to create a neural network for playing Tetris. I'm using evolutionary algorythms for it's learning, so the behavior that I need to get from the neural network is the following:

  1. Create a RNN model
  2. Give it a single input (current field state) and take an answer that takes into account previous moves
  3. Use the neural network response to make a move in the game
  4. Get new game field state, pass it to the neural network, repeat from point (2)

The problem is that on the same inputs I get the same outputs, which indicates that the RNN does not take into account its previous actions:

model = keras.Sequential(
        [
            layers.SimpleRNN(10, return_sequences=True) ,
            layers.SimpleRNN(10, return_sequences=True) ,
            layers.Dense(3, input_dim =10, activation='softmax')
        ]
    )

    x1 = np.random.rand(1, 1, inp_size)
    
    y1 = model(x1)
    y2 = model(x1)
    y3 = model(x1)

    print(y1)
    print(y2)
    print(y3)

Output:

tf.Tensor([[[0.3274583  0.42117843 0.25136325]]], shape=(1, 1, 3), dtype=float32)
tf.Tensor([[[0.3274583  0.42117843 0.25136325]]], shape=(1, 1, 3), dtype=float32)
tf.Tensor([[[0.3274583  0.42117843 0.25136325]]], shape=(1, 1, 3), dtype=float32)

So, when i do the same thing, but feed input as a sequence packed to tensor:

x = np.ones((1, 3, inp_size))
y = model(x)
print(y)

I get different results on the same inputs:

tf.Tensor(
[[[0.23801968 0.4496906  0.31228966]
  [0.30908147 0.3184522  0.37246636]
  [0.2849004  0.5014079  0.21369171]]], shape=(1, 3, 3), dtype=float32)

How can I create a RNN model which will not need a full match history as an input sequence to take proper decisions? The question is about opportunity to force network to work step by step, as if processing the sequence, but at the same time receiving only a SINGLE element at the input and immediately giving an answer (since the n-th element of the input sequence depends on the n-1 network output, and we cannot get it until previous output will not be processed by the game)

New contributor
Nick Zorander is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$
7
  • $\begingroup$ why note use Pytorch? It outputs the full history out of the box $\endgroup$ – Alex Feb 22 at 13:03
  • $\begingroup$ @Alex Is it possible to get the behavior I described above from Pytorch network? Is it as easy as Keras? $\endgroup$ – Nick Zorander Feb 22 at 20:55
  • $\begingroup$ Pytorch is not as easy as Keras, but the payoff is higher. Specifically, for an input size (1, C, H1) (batch size=1, sequence length=C, hidden dimensions=H1), LSTM outputs full history size (1, C, H2), last hidden state size (1, 1, H2) and the last cell state (1, 1, H2) $\endgroup$ – Alex Feb 22 at 23:07
  • $\begingroup$ Actually, Keras does exactly the same thing, including cell state. The question is about opportunity to force network to work step by step, as if processing the sequence, but at the same time receiving only a SINGLE element at the input and immediately giving an answer (since the n-th element of the input sequence depends on the n-1 network output, and we cannot get it until previous output will not be processed by the game) $\endgroup$ – Nick Zorander Feb 23 at 9:48
  • $\begingroup$ You can filter the whole history through a linear layer step-by-step to get n outputs, 1/step $\endgroup$ – Alex Feb 23 at 12:47

Your Answer

Nick Zorander is a new contributor. Be nice, and check out our Code of Conduct.

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.