# Penalize Neural Network Common Output

To practice reinforcement learning, I have made a neural network class and have been trying to teach it to play a toy system of Pokemon.

There are three types of pokemon, and each pokemon has access to a move called tackle and a move of its own type. When using a move of its own type, that move gets stronger.

For people who aren't aware, this game has turns, and each turn consists of 2 rounds: two actors both choose the moves for their pokemon and then the pokemon perform those moves, with the faster one moving first in this section. Each move a pokemon does reduces the opponent's health by a certain amount depending on the strength of the move. The game ends when one of the two pokemon loses all of its health.

The neural network architecture is a fully-connected feed-forward network with backpropagation. The problem persists using both sigmoid and relu activation functions. The input is a 10-length vector, a one-hot of the player's type (3), of the enemy's type (3) and a vector of the moves at the current pokemon's disposal (4). The output is a vector of length 4, where the highest (legal) value is considered the index of the move the pokemon will choose. In the beginning of training, random moves are chosen and then slowly not for exploration/exploitation. The change of Q each time is adding the reward (relative amount of health lost compared to full of the opponent) to the Q-value of the move chosen and then giving the output vector with the updated value on the one move as the "true" Q value.

While the pokemon SHOULD be choosing the move that does the most damage, they are for the most part choosing tackle. I think that it is due to tackle being given rewards more often than the other moves, as all three pokemon can use tackle.

What is the proper thing to do to fix this error in the training process? Do I introduce a factor to scale the reward by how many pokemon get the move and the probability that it will be randomly chosen?

Thank you!

Edit: introducing a 1/10 reward constraint for tackle did not solve the problem. It now only most of the times results in tackle being the most chosen move.

• so... Is tackle indeed the most effective move? How do the other moves compare to it in terms of damage/turns or some other metric? – Ingolifs Oct 15 '18 at 4:29
• Ive played around with it. The other moves do 50% more damage normally, but I have changed that ratio of damage to 10:1 and still saw some problems with skew. – Jeff Oct 15 '18 at 6:08
• I am not quite sure what you are trying to do. Are you trying to use Reinforcement Learning? Which RL algorithm are you using? It seems that your input contains information of both players. Why the actions as inputs to your network? Could you give more details if you are following a specific method or are experimenting with your own technique? – Constantinos Oct 18 '18 at 23:19