Skip to main content

Questions tagged [dqn]

The tag has no usage guidance, but it has a tag wiki.

Filter by
Sorted by
Tagged with
11 votes
2 answers
3k views

How does Implicit Quantile-Regression Network (IQN) differ from QR-DQN?

For several months I browsed the internet hoping to find a user-friendly explanation of the Implicit Quantile Regression Network (IQN). But, it seems there is none at all. How does IQN differ from ...
Kari's user avatar
  • 2,736
10 votes
2 answers
18k views

what is difference between the DDQN and DQN?

I think I did not understand what is the difference between DQN and DDQN in implementation. I understand that we change the traget network during the running of DDQN but I do not understand how it is ...
user10296606's user avatar
  • 1,844
7 votes
3 answers
4k views

Why random sample from replay for DQN?

I'm trying to gain an intuitive understanding of deep reinforcement learning. In deep Q-networks (DQN) we store all actions/environments/rewards in a memory array and at the end of the episode, "...
ZAR's user avatar
  • 203
7 votes
1 answer
4k views

What are the effects of clipping the reward in stability?

I am looking for stabilizing my results of DQN, I found clipping is one technique to do it but I did not understand it completely! 1- what are the effects of clipping the reward, clipping the ...
user10296606's user avatar
  • 1,844
6 votes
2 answers
2k views

How to choose between discounted reward and average reward?

How to select between average reward and discounted reward? And when average reward is more effective in comparison with discounter reward and when vice versa is correct? Is is possible to use both ...
user10296606's user avatar
  • 1,844
5 votes
1 answer
1k views

Difference between advantages of Experience Replay in DQN2013 paper

I've been re-reading the Playing Atari with Deep Reinforcement Learning (2013) paper. It lists three advantages of experience replay: This approach has several advantages over standard online Q-...
seungjaeryanlee's user avatar
4 votes
2 answers
5k views

Agent always takes a same action in DQN - Reinforcement Learning

I have trained an RL agent using DQN algorithm. After 20000 episodes my rewards are converged. Now when I test this agent, the agent is always taking the same action , irrespective of state. I find ...
chink's user avatar
  • 555
4 votes
1 answer
2k views

Why does exploration in DQN not lead to instability?

Why does action exploration in DQN not lead to instability? I see in DQN algorithms, that it selects random actions even after some iterations. My question is how does this approach not lead to ...
user10296606's user avatar
  • 1,844
4 votes
1 answer
648 views

How to implement clipping the reward in DQN in keras

How to implement clipping the reward in DQN in keras? especially how to implement clipping the reward? Is this pseudo code correct: ...
user10296606's user avatar
  • 1,844
4 votes
1 answer
2k views

What is a minimal setup to solve the CartPole-v0 with DQN?

I solved the CartPole-v0 with a CEM agent pretty easily (experiments and code), but I struggle to find a setup which works with DQN. Do you know which parameters should be adjusted so that the mean ...
Martin Thoma's user avatar
4 votes
1 answer
1k views

DQN fails to find optimal policy

Based on DeepMind publication, I've recreated the environment and I am trying to make the DQN find and converge to an optimal policy. The task of an agent is to learn how to sustainably collect apples ...
macwiatrak's user avatar
3 votes
2 answers
513 views

How exactly does DQN learn?

I created my custom environment in gym, which is a maze. I use a DQN model with ...
Marci's user avatar
  • 31
3 votes
1 answer
963 views

Deep reinforcement learning on changing data sizes

I have a game that I want to build a model that will learn to play the game. Yet, the environment output is two lists that represent the location and number of soldiers of the user and Opponent. The ...
nimrod feldman's user avatar
3 votes
1 answer
2k views

Difference between Dueling DQN and Double DQN?

I have read some articles, but still can not figure out the difference between the Dueling DQN and Double DQN? What exactly is the difference between them? Also, Does Dueling DQN need to be built on ...
Edamame's user avatar
  • 2,765
3 votes
1 answer
5k views

Evaluating a trained Reinforcement Learning Agent?

I am new to reinforcement learning agent training. I have read about PPO algorithm and used stable baselines library to train an agent using PPO. So my question here is how do I evaluate a trained RL ...
chink's user avatar
  • 555
3 votes
1 answer
363 views

Clamping Q function to it's theoretical maximum, yes or no?

I'm implementing DQN algorithm from scratch on MountainCar simulation. I'm using a setup of $reward = 1.0$ when car hits the flag, and $0$ otherwise. Reward decay factor is set to $\gamma=0.99$. ...
omittones's user avatar
  • 133
3 votes
1 answer
875 views

What is wrong with this reinforcement learning environment ?

I'm working on below reinforcement learning problem: I have bottle of fix capacity (say 5 liters). At the bottom of bottle there is cock to remove water. The distribution of removal of water is not ...
Krishna Nevase's user avatar
3 votes
1 answer
244 views

Is it possible to solve Rubik's cube using DQN?

I'm trying to solve Rubik's cube using deep learning and I came across with DQN, so I decided to give it a try. I developed all the code and started training but I got this results: Loss goes up and ...
Javier Jiménez de la Jara's user avatar
3 votes
0 answers
725 views

Deep Reinforcement Learning for dynamic pricing

I am trying to implement a Deep Q Network model for Dynamic pricing in Logistics. I can define State Space (Origin, Destination, type of the shipment, customer, Type of the product, Commodity of the ...
Karthik Rajkumar's user avatar
2 votes
1 answer
5k views

DQN with decaying epsilon

I'm new to reinforcement learning. I'm studying DQN with decaying epsilon. I came across such example: EPISODES = 91 GAMMA = 0.2 EPSILON_DECAY = 0.999 MIN_EPSILON = 0.01 MAX_EPSILON = 1 My questions ...
Martin's user avatar
  • 123
2 votes
1 answer
6k views

Policy Gradient with continuous action space

How to apply reinforce/policy-gradient algorithms for continuous action space. I have learnt that one of the advantages of policy gradients is , it is applicable for continuous action space. One way I ...
chink's user avatar
  • 555
2 votes
3 answers
346 views

Which is more important - stable training results or good test results?

Which is more important - stable training results or good test results? For instance, is obtaining an unstable training accuracy in different epochs, but good test accuracy better? Or is obtaining a ...
user10296606's user avatar
  • 1,844
2 votes
1 answer
187 views

How to Form the Training Examples for Deep Q Network in Reinforcement Learning?

Trying to pick up basics of reinforcement learning by self-study from some blogs and texts. Forgive me if the question is too basic and different bits that I understand are a bit messy, but even after ...
Della's user avatar
  • 335
2 votes
1 answer
1k views

When should the last action be included in the state in reinforcement learning?

I am having some confusion as to whether the action should be included as part of the state input to an agent in a reinforcement learning setting (state-action pair). As from my observation, this is ...
user91315's user avatar
2 votes
1 answer
1k views

Why is Distributional DQN faster than vanilla DQN?

Recently I learned about Distributional approach to RL, which is a quite fascinating and break--through algorithm. I have 2 questions: What is it that makes it perform so much better during runtime ...
Kari's user avatar
  • 2,736
2 votes
1 answer
193 views

How do I build a DQN which selects the correct objects in an environment based on the environment state?

I have an environment with 4 objects in it. All of these objects can either be selected or not selected. So the actions taken by my DQN should look like - ...
Atharva Nimbalkar's user avatar
2 votes
1 answer
363 views

Representation of state space, action space and reward system for RL porblem

I am trying to solve the problem of an agent dynamically discovering(start with no information about the environment) the environment and to explore as much of the environment as possible without ...
Incompleteness's user avatar
2 votes
1 answer
825 views

Would Deep Q Learning work for a finite horizon problem?

I want to apply Deep Q Learning to a problem, which has a clear finite horizon definition, like: $$V(s) = \mathbb{E}[r_1 + r_2]$$ Since the horizon is finite, I do not use reward discounting. My ...
Ufuk Can Bicici's user avatar
2 votes
1 answer
572 views

Having a reward structure which gives high positive rewards compared to the negative rewards

I am training an RL agent using PPO algorithm for a control problem. The objective of the agent is to maintain temperature in a room. It is an episodic task with episode length of 9 hrs and step size(...
chink's user avatar
  • 555
2 votes
1 answer
66 views

"Each agent was evaluated every 250,000 training frames for 135,000 validation frames" What does this sentences stands for? in DQN nature paper?

In nature paper of DQN by DeepMind, DQN is compared to linear function but they does not said what is this linear function? They compared with some linear functions? 0- What is the meaning of this ...
user10296606's user avatar
  • 1,844
2 votes
1 answer
136 views

How we can have RF-QLearning or SVR-QLearning (Combine these algorithm with a Q-Learning )

How we can have RF-QLearning or SVR-QLearning (Combine these algorithm with a Q-Learning )? I want to replace the DNN section of Qlearning with a RF or SVR but the problem is that there is no clear ...
user10296606's user avatar
  • 1,844
2 votes
1 answer
634 views

Is the neural network in DQN used to learn like a supervised model?

Is the neural network in DQN used to learn like a supervised model?
user10296606's user avatar
  • 1,844
2 votes
1 answer
159 views

Dimensionality of the target for DQN agent training

From what I understand, a DQN agent has as many outputs as there are actions (for each state). If we consider a scalar state with 4 actions, that would mean that the DQN would have a 4 dimensional ...
Dhoop's user avatar
  • 21
2 votes
0 answers
225 views

How to resolve the instability of average reward per episode in training of DQN (Deep Q-Network)?

what is shown when average reward per episode in training is unstable? If there is big difference between average reward per episode and final reward by test section, what we can say? For instance in ...
user10296606's user avatar
  • 1,844
2 votes
1 answer
599 views

How to give rewards to actions in RL?

I'm working on below reinforcement learning problem: I have bottle of fix capacity (say 5 liters). At the bottom of bottle there is cock to remove water. The distribution of removal of water is not ...
Krishna Nevase's user avatar
1 vote
1 answer
2k views

RL - Weighthing negative rewards [closed]

Let's consider that I give an agent a reward of -1 (minimum reward) every time it performs an action which leads to the premature end of the episode (i.e., the agent dies). Besides, I also give a ...
nil's user avatar
  • 88
1 vote
2 answers
87 views

Free a bit of RAM space

Here is Colab Notebook After 1500 episodes if batch_size=256, the RAM crashed. With Colab, I have the equivalent of 25.5 gigs of RAM. Is it normal? Or I don't have ...
jgauth's user avatar
  • 131
1 vote
1 answer
1k views

Different results every time I train a reinforcement learning agent

I am training an RL agent for a control problem using PPO algorithm. I am using stable-baselines library for it. The objective of an agent is to maintain a temperature of 24 deg in a zone and it ...
chink's user avatar
  • 555
1 vote
1 answer
193 views

Cartpole - Number of layers and neurons - model hyperparameters

Can anyone please suggest me how to arrive to the best optimal values for number of layers, number of neurons parameters of the deep learning model in DDQN algorithm for cartpole problem. As input and ...
vimala's user avatar
  • 21
1 vote
1 answer
186 views

In DQN, why not use target network to predict current state Q values?

In DQN, why not use target network to predict current state Q values, and not only next state q values? In doing a basic dq learning algorithm with nn from scratch, with replay memory, and minibatch ...
Lorenzo Tinfena's user avatar
1 vote
1 answer
215 views

How to formulate reward of an rl agent with two objectives

I have started learning reinforcement learning and trying to apply it for my use case. I am developing an rl agent which can maintain temperature at a particular value, and minimize the energy ...
chink's user avatar
  • 555
1 vote
1 answer
145 views

Is DQN limited to working with only image frames?

I have few questions about Deep Q Network. Does DQN only accept image frames as input? I have never hear (read) a paperwork where it doesn't use image frames. If the first is a No, then does image ...
ElLoco's user avatar
  • 13
1 vote
1 answer
263 views

Will reinforcement learning work if states wont get repeated again?

I am working on a information retrieval model where the user enters a query and the model has to retrieve 3 most relevant FAQ pairs.I am collecting implicit feedback in terms of page clicks etc.What I ...
Charan Reddy's user avatar
1 vote
1 answer
453 views

What is difference between final episodes of training and test in DQN?

What is difference between running in final episode of training mode and running in test mode in DQN? Is there any difference more than after training and tune the hyper-parameters, we test for one ...
user10296606's user avatar
  • 1,844
1 vote
1 answer
462 views

Reinforcement Learning with static state

Can Q Learning work with a static state for each step? What I mean by that is that the actions do not influence the following state at all. The episodes just iterate over the same data over and over ...
Pit Wegner's user avatar
1 vote
0 answers
34 views

Building an engine for a chess-like game

I'm working on building an AI for a chess-like game. I've implemented a Monty Carlo Tree Search (for the early game) and Rainbow DQN (for the mid to late game), and will be implementing Alpha Beta ...
william paine's user avatar
1 vote
1 answer
63 views

Can Online DQN model overfit?

I am new in the area of RL and currently trying to train an online DQN model. Can an online model overfit since its always learning? and how can I tell if that happens?
user125612's user avatar
1 vote
0 answers
208 views

DQN CartPole-v1 neural network doesn't optimize

I'm doing my first dnq algorithm, I'm trying to build a dnq agent, and neural network from scratch, but it seems that neural network doesn't optimize, I did 2 hidden layers, with ReLU, and the output ...
Lorenzo Tinfena's user avatar
1 vote
0 answers
271 views

DQN DDQN and 3DQN differences?

I'm doing a course on reinforcement learning, and one of our tasks is to implement an agent on the Lunar lander continuous V2 environment from openAI gym. In order to solve the continuous problem, I ...
user113367's user avatar
1 vote
0 answers
11 views

Implementing DQN - Pylessons.com article - Queries [closed]

I have followed the below link to implement DQN Algorithm https://pylessons.com/CartPole-reinforcement-learning/ Can someone explain me: Why do we need to have else condition in act function during ...
vimala's user avatar
  • 21