I am using Reinforcement Learning to teach an AI an Austrian Card Game with imperfect information called Schnapsen. For different states of the game, I have different neural networks (which use different features) that calculate the value/policy. I would like to try using RNNs, as past actions may be important to navigate future decisions.
However, as I use multiple neural networks, I somehow need to constantly transfer the hidden state from one RNN to another one. I am not quite able to do that, especially during training I don't know how to make backpropagation through time work. I am grateful for any advice or links to related papers/blogs!
I am currently working with Flux in Julia, but I am also willing to switch to Tensorflow or Pytorch in Python.