1
$\begingroup$

I am using Reinforcement Learning to teach an AI an Austrian Card Game with imperfect information called Schnapsen. For different states of the game, I have different neural networks (which use different features) that calculate the value/policy. I would like to try using RNNs, as past actions may be important to navigate future decisions.

However, as I use multiple neural networks, I somehow need to constantly transfer the hidden state from one RNN to another one. I am not quite able to do that, especially during training I don't know how to make backpropagation through time work. I am grateful for any advice or links to related papers/blogs!

I am currently working with Flux in Julia, but I am also willing to switch to Tensorflow or Pytorch in Python.

$\endgroup$
2
  • $\begingroup$ transfering hidden states is hacky and as far as I know, there is no theoretical basis to support it. It is a hack $\endgroup$
    – Nikos M.
    Commented Oct 11, 2021 at 15:57
  • $\begingroup$ It will be helpful if you could briefly describe the task. How many players, competitive/cooperative nature of the game etc. Also why are you using different RNNs for different states of the game? Could you provide more details also on how you are using these networks and what you want as an end result? $\endgroup$ Commented Oct 11, 2021 at 16:49

1 Answer 1

0
$\begingroup$

These are a few approaches I found in the research field that combines both RNN and Reinforcement Learning that looks promising

  • Reinforcement learning with LSTM networks
  • Reinforcement learning with RNN
  • Hybrid RNN approach

Research paper links

  1. A Reinforcement Learning and Recurrent Neural Network Based Dynamic User Modeling System
  2. Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation
  3. The state of mind: Reinforcement learning with recurrent neural networks
  4. Reinforcement learning by backpropagation through an LSTM model/critic
  5. Hybrid RNN

Note: The following paper seems to be the only one verified in the industry and applied to an industrial problem D. Prokhorov, Toward effective combination of off-line and on-line training in ADP framework, in: Proceedings of the IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL), Honolulu, HI, pp. 268–271, 2007

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.