Questions tagged [q-learning]

A model-free reinforcement learning technique.

Filter by
Sorted by
Tagged with
44
votes
2answers
53k views

What is "experience replay" and what are its benefits?

I've been reading Google's DeepMind Atari paper and I'm trying to understand the concept of "experience replay". Experience replay comes up in a lot of other reinforcement learning papers (...
8
votes
1answer
7k views

RL Advantage function why A = Q-V instead of A=V-Q?

In RL Course by David Silver - Lecture 7: Policy Gradient Methods, David explains what an Advantage function is, and how it's the difference between Q(s,a) and the V(s) Preliminary, from this post: ...
8
votes
1answer
8k views

Reinforcement learning: decreasing loss without increasing reward

I'm trying to solve OpenAI Gym's LunarLander-v2. I'm using the Deep Q-Learning algorithm. I have tried various hyperparameters, but I can't get a good score. Generally the loss decreases over many ...
5
votes
2answers
3k views

Why random sample from replay for DQN?

I'm trying to gain an intuitive understanding of deep reinforcement learning. In deep Q-networks (DQN) we store all actions/environments/rewards in a memory array and at the end of the episode, "...
3
votes
1answer
51 views

Exploration in Q learning: Epsilon greedy vs Exploration function

I am trying to understand how to make sure that our agent explores the state space enough before exploiting what it knows. I am aware that we use epsilon-greedy approach with a decaying epsilon to ...
3
votes
1answer
776 views

DQN fails to find optimal policy

Based on DeepMind publication, I've recreated the environment and I am trying to make the DQN find and converge to an optimal policy. The task of an agent is to learn how to sustainably collect apples ...
2
votes
1answer
93 views

Experience Replay, must return minibatch back to Memory Bank?

During Experience Replay, we are randomly gathering a minibatch from the Memory bank. We then use the minibatch to correct our NeuralNetwork q-value function approximator. When done, should we return ...
1
vote
1answer
893 views

DQN - target values vs action values?

I'm trying to understand the difference between target-values and action-values in Deep Q Networks. From what I understand, action-value tries to approximate the reward of a given action (at some ...