I am trying to create an AI to play a 3D melee fighting game using an LSTM. The NN receives as input the relative positions of the enemies' joints (e.g. head, hands, legs) and should output the correct movements for each timestep. Currently, I am using a vanilla feed-forward NN made with Keras and it performs well, but it needs information from multiple timesteps to be more effective.
I am now trying to implement an LSTM for this problem. I want to receive an output at each timestep rather than feeding in a whole sequence of data before receiving output.
I believe that a stateful LSTM is suited for this task, but I am unsure of how to train it and when to reset the state. Ideally, the AI would have no fixed number of timesteps of memory and would just be able to continuously look back as it's making predictions in real time for each timestep, without having its state reset abruptly during fights.
My training data is one long, continuous sequence of enemy positions (input) and keystrokes (output) for each timestep. How would I structure and train an LSTM to learn from this data to make real time predictions at each timestep using Keras?