I've creating a very simple game, The board is an array of size 6. 0 is "empty cell" , 5 is "Goal", 8 is "player location" [8 0 0 5 0 0] for example means the agent needs to move 2 "right" to win.
hitting a "5" ends the game with reward +1 hitting a wall ends the game with reward -1 other moves have reward 0.
the "state" is simply the board, 6-long vector that can be 0\8\5
When i built a regular Q-Table based agent the game converged (to a good result) very fast.
The problem is that the DQN (Deep Q network) agent never converged. i sampled the network after a few thousand iterations and saw that the network for all inputs always has the same result.
The actual result keeps changing, each replay of the memory, but at each point if i ask to predict, no matter the input, the output will be "LEFT: X, RIGHT: Y". (X and Y are numeric values). the network ignores the input. any suggestions why?
Appendix: Hyperparams: batch-size = 64 memory = 100 (tried 1000 as well( gamma = 0.99 # discount rate epsilon_min = 0.05 epsilon_decay = 0.95 epsilon_start = 1.0 learning rate = 0.01 / 0.1 / 1.0
the network is ( i tried several):
# Neural Net for Deep-Q learning Model model = K.models.Sequential() model.add(K.layers.Dense(8, input_dim=self.state_size, activation='relu')) model.add(K.layers.Dense(8, activation='relu')) model.add(K.layers.Dense(self.action_size, activation='linear')) model.compile(loss='mse', optimizer=K.optimizers.Adam(lr=self.learning_rate))