Transferring the hidden state of a RNN to another RNN

Question

I am using Reinforcement Learning to teach an AI an Austrian Card Game with imperfect information called Schnapsen. For different states of the game, I have different neural networks (which use different features) that calculate the value/policy. I would like to try using RNNs, as past actions may be important to navigate future decisions.

However, as I use multiple neural networks, I somehow need to constantly transfer the hidden state from one RNN to another one. I am not quite able to do that, especially during training I don't know how to make backpropagation through time work. I am grateful for any advice or links to related papers/blogs!

I am currently working with Flux in Julia, but I am also willing to switch to Tensorflow or Pytorch in Python.

Answer 1

These are a few approaches I found in the research field that combines both RNN and Reinforcement Learning that looks promising

• Reinforcement learning with LSTM networks
• Reinforcement learning with RNN
• Hybrid RNN approach

Research paper links

1. A Reinforcement Learning and Recurrent Neural Network Based Dynamic User Modeling System
2. Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation
3. The state of mind: Reinforcement learning with recurrent neural networks
4. Reinforcement learning by backpropagation through an LSTM model/critic
5. Hybrid RNN

Note: The following paper seems to be the only one verified in the industry and applied to an industrial problem D. Prokhorov, Toward effective combination of off-line and on-line training in ADP framework, in: Proceedings of the IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL), Honolulu, HI, pp. 268–271, 2007