Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [dqn]

The tag has no usage guidance.

0
votes
1answer
35 views

Q-Learning experience replay: how to feed the neural network?

I'm trying to replicate the DQN Atari experiment. Actually my DQN isn't performing well; checking another one's codes, I saw something about experience replay which I don't understand. First, when you ...
2
votes
1answer
90 views

DQN fails to find optimal policy

Based on DeepMind publication, I've recreated the environment and I am trying to make the DQN find and converge to an optimal policy. The task of an agent is to learn how to sustainably collect apples ...
0
votes
0answers
12 views

Is reward accumulated during a play iteration when performing SARSA?

I've been having issue with getting my DQN to converge to a good solution for snake. Regardless of the different types of reward functions I've tried, it seems that the snake is indefinitely going ...
0
votes
0answers
10 views

Is DQN independent from type of input distribution?Why?

Can we say: Since DQN is online learning, it is independent of type of input distribution?
2
votes
0answers
51 views

Deep Reinforcement Learning for dynamic pricing

I am trying to implement a Deep Q Network model for Dynamic pricing in Logistics. I can define State Space (Origin, Destination, type of the shipment, customer, Type of the product, Commodity of the ...
4
votes
1answer
44 views

how to choose between discounted reward and average reward

how to select between average reward and discounted reward? And when average reward is more effective in comparison with discounter reward and when vice versa is correct? -Is is possible to use both ...
0
votes
0answers
10 views

Is DQN a SMART (Semi Markov Average Reward Technique)?

Is DQN a SMART (Semi Markov Average Reward Technique) algorithm?
1
vote
0answers
21 views

What is the meaning of the Variant Q-learning and To what INPUT and OUTPUT refer? in Abstract of DeepMind DQN paper 2013

-INPUT and OUTPUT OF ATARI DQN: In the abstract paragraph of the DQN work by DeepMind https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf it has written: " We present the first deep learning model to ...
0
votes
0answers
5 views

effects of number of running time periods on examining the DQN quality?

What are the effects of the number of running time periods on examining the DQN quality? I mean "T": time periods of training and testing. If there is not an obligation to set it to a value in the ...
0
votes
1answer
55 views

RL - Weighthing negative rewards [closed]

Let's consider that I give an agent a reward of -1 (minimum reward) every time it performs an action which leads to the premature end of the episode (i.e., the agent dies). Besides, I also give a ...
5
votes
1answer
288 views

How does Implicit Quantile-Regression Network (IQN) differ from QR-DQN?

For several months I browsed the internet hoping to find a user-friendly explanation of the Implicit Quantile Regression Network (IQN). But, it seems there is none at all. How does IQN differ from ...
1
vote
1answer
43 views

Will reinforcement learning work if states wont get repeated again?

I am working on a information retrieval model where the user enters a query and the model has to retrieve 3 most relevant FAQ pairs.I am collecting implicit feedback in terms of page clicks etc.What I ...
1
vote
0answers
69 views

comparison of linear Q-learning and DQN

I saw in DQN nature paper 2015 https://www.nature.com/articles/nature14236(Extended Data Table 4) some comparisons between DQN and linear Q-learning. The ratio ...
0
votes
1answer
71 views

is it acceptable if the reward of test of DQN is lower than reward of training of DQN in minimization problem?

if we train a DQN over 40000-60000 episodes for 500 time steps. The mean of reward during last 100 training steps is about 1.1 times of reward during the test process. Environment is stochastic! The ...
0
votes
0answers
52 views

how to calculate number of frames in epsilon greedy with decay rate?

If starting epsilon is alpha and end epsilon is beta in epsilon greedy algorithm. discount rate is gamma and epsilon decay is lambda, how to calculate the F: number of frames to reach from alpha to ...
0
votes
0answers
105 views

What are the reasons of select a optimizer to be SGD or Adam in DQN?Why?

I saw several comparison between SGD, RMSPROP and ADAM but what I am looking for is their comparsion in DQN algorithm? What is best to select as optimizer SGD or Adam in DQN? Why? Please check the ...
2
votes
1answer
47 views

“Each agent was evaluated every 250,000 training frames for 135,000 validation frames” What does this sentences stands for? in DQN nature paper?

In nature paper of DQN by DeepMind, DQN is compared to linear function but they does not said what is this linear function? They compared with some linear functions? 0- What is the meaning of this ...
3
votes
1answer
255 views

How to implement clipping the reward in DQN in keras

How to implement clipping the reward in DQN in keras? especially how to implement clipping the reward? Is this pseudo code correct: ...
3
votes
1answer
109 views

What are the effects of clipping the reward in stability?

I am looking for stabilizing my results of DQN, I found clipping is one technique to do it but I did not understand it completely! 1- what are the effects of clipping the reward, clipping the ...
1
vote
1answer
72 views

What is difference between final episodes of training and test in DQN?

What is difference between running in final episode of training mode and running in test mode in DQN? Is there any difference more than after training and tune the hyper-parameters, we test for one ...
1
vote
0answers
137 views

DQN cannot learn or converge

I have implemented a DQN using keras. The task is to collect the circles and avoid the red circle and crosses. The associated rewards are +5, -5 and 0 otherwise. if the agent go out of the board, the ...
3
votes
2answers
1k views

what is difference between the DDQN and DQN?

I think I did not understand what is the difference between DQN and DDQN in implementation. I understand that we change the traget network during the running of DDQN but I do not understand how it is ...
2
votes
3answers
31 views

which of stable training results or good test results is more important?

which of stable training results or good test results are more important? For instance obtaining an unstable training accuracy in different episodes but good test accuracy is better or obtaining a ...
2
votes
0answers
92 views

How to resolve the instability of average reward per episode in training of DQN (Deep Q-Network)?

what is shown when average reward per episode in training is unstable? If there is big difference between average reward per episode and final reward by test section, what we can say? For ...
2
votes
1answer
71 views

Why does exploration in DQN not lead to instability?

Why does action exploration in DQN not lead to instability? I see in DQN algorithms, that it selects random actions even after some iterations. My question is how does this approach not lead to ...
2
votes
1answer
39 views

How we can have RF-QLearning or SVR-QLearning (Combine these algorithm with a Q-Learning )

How we can have RF-QLearning or SVR-QLearning (Combine these algorithm with a Q-Learning )? I want to replace the DNN section of Qlearning with a RF or SVR but the problem is that there is no clear ...
1
vote
1answer
58 views

Is the neural network in DQN used to learn like a supervised model?

Is the neural network in DQN used to learn like a supervised model?
1
vote
0answers
346 views

Why is my loss function for DQN converging too quickly?

I'm still relatively new to deep learning and am experiencing an issue that I can't seem to find a solution/explanation for. I've developed a DQN model in tensorflow, as described by DeepMind, and am ...
2
votes
1answer
95 views

What is wrong with this reinforcement learning environment ?

I'm working on below reinforcement learning problem: I have bottle of fix capacity (say 5 liters). At the bottom of bottle there is cock to remove water. The distribution of removal of water is not ...
1
vote
1answer
131 views

How to give rewards to actions in RL?

I'm working on below reinforcement learning problem: I have bottle of fix capacity (say 5 liters). At the bottom of bottle there is cock to remove water. The distribution of removal of water is not ...
1
vote
0answers
83 views

Deep Q-Learning with large number of actions

I'm using DQN with large number of actions in [0, 10000, step = 1000]. This means I have an action space of size 11 (including 0 and 10000). Action space is still discrete. My problem is that, instead ...
3
votes
1answer
200 views

Difference between advantages of Experience Replay in DQN2013 paper

I've been re-reading the Playing Atari with Deep Reinforcement Learning (2013) paper. It lists three advantages of experience replay: This approach has several advantages over standard online Q-...
1
vote
1answer
203 views

Why is Distributional DQN faster than vanilla DQN?

Recently I learned about Distributional approach to RL, which is a quite fascinating and break--through algorithm. I have 2 questions: What is it that makes it perform so much better during runtime ...
1
vote
0answers
32 views

Can a DQN output a float result? [closed]

I'm a newbie of Deep Q Learning. After read some papers and tutorials on the web, I tried to train a DQN to control a game using TensorFlow. The input is the screenshoot of the game. The output is an ...
1
vote
0answers
117 views

Experience Replay Explain

I have read many blog articles, research papers and watched many youtube videos, but it seems it is hard to find why experience replay is efficient. I know that the experience replay stores (state, ...
1
vote
1answer
95 views

Reinforcement Learning with static state

Can Q Learning work with a static state for each step? What I mean by that is that the actions do not influence the following state at all. The episodes just iterate over the same data over and over ...
3
votes
1answer
904 views

Why random sample from replay for DQN?

I'm trying to gain an intuitive understanding of deep reinforcement learning. In deep Q-networks (DQN) we store all actions/environments/rewards in a memory array and at the end of the episode, "...
3
votes
1answer
108 views

Clamping Q function to it's theoretical maximum, yes or no?

I'm implementing DQN algorithm from scratch on MountainCar simulation. I'm using a setup of $reward = 1.0$ when car hits the flag, and $0$ otherwise. Reward decay factor is set to $\gamma=0.99$. ...
1
vote
1answer
1k views

What is a minimal setup to solve the CartPole-v0 with DQN?

I solved the CartPole-v0 with a CEM agent pretty easily (experiments and code), but I struggle to find a setup which works with DQN. Do you know which parameters should be adjusted so that the mean ...