# Major drop in DeepQNetwork agent after certain number of iteration

I've built a game with an agent moving around trying to collect gold (+10 reward) and it dies when it hits a wall (-100, terminal condition) and -1 for any other step that is not gold nor wall. My action space is 4 (U, D,R,L) and my state is the sight he has. [More details in the end of the message]

The problem is it improves for a few hundred\thousands (depending on the model) iterations and then it drops to into a mode it simply runs into a wall, never improving from there.

This is my model:

            self.model = K.models.Sequential()
self.model.compile(loss="mse", optimizer=optimizer)


I've tried debugging it, and encountered the following behaviour:

During the phase in which i experience a replay i pulled a "state", with reward -100, a terminal state, in which the agent chose to run into a wall.

when i model.predict(state) I got: 39.8269 18.1576 21.3698 15.1639 The action in the memory is 2 with reward -100, so i changed to: 39.8269 18.1576 -100 15.1639

And let it train, i re-predicted and got: 40.0833 18.1578 21.1512 15.2231

Only a minor change for such a big error isn't it? Could this be the reason for my drop in performance? perhaps there is a different reason i'm missing?

Appendix: The game has a board of NxN the agent starts in the center (rounded up) Every other cell has a gold coin. When an agent steps into a cell with a gold coin he gets +10 and the coin is gone. When an agent steps into an empty cell he gets -1 reward. When an agent steps outside the array he gets reward -100 and the game ends. if the agent doesn't find gold in 6 turns the game ends and the agent gains -1.

The input layer is [0:3] are the last action performed in a one-hot-encoding [5] is the moves until starvation. [5:8] the square above the agent,below,right and left.[-1 wall, 1 empty cell, 2 gold coin] [9:12] are the squares in diagonal to the agent (UR,UL,DR,DL)

Thank you!