To practice reinforcement learning, I have made a neural network class and have been trying to teach it to play a toy system of Pokemon.
There are three types of pokemon, and each pokemon has access to a move called tackle and a move of its own type. When using a move of its own type, that move gets stronger.
For people who aren't aware, this game has turns, and each turn consists of 2 rounds: two actors both choose the moves for their pokemon and then the pokemon perform those moves, with the faster one moving first in this section. Each move a pokemon does reduces the opponent's health by a certain amount depending on the strength of the move. The game ends when one of the two pokemon loses all of its health.
The neural network architecture is a fully-connected feed-forward network with backpropagation. The problem persists using both sigmoid and relu activation functions. The input is a 10-length vector, a one-hot of the player's type (3), of the enemy's type (3) and a vector of the moves at the current pokemon's disposal (4). The output is a vector of length 4, where the highest (legal) value is considered the index of the move the pokemon will choose. In the beginning of training, random moves are chosen and then slowly not for exploration/exploitation. The change of Q each time is adding the reward (relative amount of health lost compared to full of the opponent) to the Q-value of the move chosen and then giving the output vector with the updated value on the one move as the "true" Q value.
While the pokemon SHOULD be choosing the move that does the most damage, they are for the most part choosing tackle. I think that it is due to tackle being given rewards more often than the other moves, as all three pokemon can use tackle.
What is the proper thing to do to fix this error in the training process? Do I introduce a factor to scale the reward by how many pokemon get the move and the probability that it will be randomly chosen?
Edit: introducing a 1/10 reward constraint for tackle did not solve the problem. It now only most of the times results in tackle being the most chosen move.