There's a popular solution to the CartPole game using Keras and Deep Q-Learning: https://keon.github.io/deep-q-learning/
But there's a line of code that's confusing, this same question has been asked in the same article and many people are confused but there's no a complete answer.
They are basically creating a main network but also a target network to try to approximate the Q function.
In this part of the code they are replaying from the buffer to train the target network:
# Sample minibatch from the memory minibatch = random.sample(self.memory, batch_size) # Extract informations from each memory for state, action, reward, next_state, done in minibatch: # if done, make our target reward target = reward if not done: # predict the future discounted reward target = reward + self.gamma * \ np.amax(self.model.predict(next_state)) # make the agent to approximately map # the current state to future discounted reward # We'll call that target_f target_f = self.model.predict(state) target_f[action] = target # Train the Neural Net with the state and target_f self.model.fit(state, target_f, epochs=1, verbose=0)
What I can't understand is this line:
target_f[action] = target
In terms of code, the
predict function is returning a numpy array of arrays, like this one for example:
target_f to access the first predicted action is understandable, but why are they using the
Thank you very much for the help!