Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [dqn]

The tag has no usage guidance.

0
votes
0answers
7 views

add a constraint to a DQN

I have a DQN (with 5 state variables) but I want to add a constraint to it such as (p<10), which p is a parameter which is not one of the state variables. However, there is a hide relation between ...
0
votes
0answers
24 views

Help in tuning hyperparameters of DQN based on resulted stats

I'm not an expert in machine learning. I'm trying to make an agent learn to play in a Gym environment that I created. I'm not asking for help in debugging the code. I'm asking for help in interpreting ...
2
votes
1answer
69 views

How does Implicit Quantile-Regression Network (IQN) differ from QR-DQN?

As a newbie, for several months I browsed the internet hoping to find a user-friendly explanation of the Implicit Quantile Regression Network (IQN). But, it seems there is none at all. How does IQN ...
0
votes
0answers
26 views

Dueling Network architecture & max Advantage

From my current understanding, the benefit of Dueling Network architecture (for RL) is that it provides a way to automatically estimate: Value: total future reward if we promise to select actions ...
1
vote
1answer
23 views

Will reinforcement learning work if states wont get repeated again?

I am working on a information retrieval model where the user enters a query and the model has to retrieve 3 most relevant FAQ pairs.I am collecting implicit feedback in terms of page clicks etc.What I ...
0
votes
0answers
9 views

what does frame mean? what is its difference in comparison with training epoch in DQN?

what does frame mean? what is its difference in comparison with training epoch in DQN? http://arxiv.org/abs/1810.07286 Also, I have seen in DQN nature paper it has written they trained on 1 ...
0
votes
0answers
10 views

What is shown when results of linear Q-learning during training is better than deep Q-learning?

What is shown when results of linear Q-learning during training is better than deep Q-learning? I have experienced during 1000 episodes and compare the results of DQN and Linear Q-learning (having ...
1
vote
0answers
39 views

comparison of linear Q-learning and DQN

I saw in DQN nature paper 2015 https://www.nature.com/articles/nature14236(Extended Data Table 4) some comparisons between DQN and linear Q-learning. The ratio ...
0
votes
1answer
28 views

is it acceptable if the reward of test of DQN is lower than reward of training of DQN in minimization problem?

if we train a DQN over 40000-60000 episodes for 500 time steps. The mean of reward during last 100 training steps is about 1.1 times of reward during the test process. Environment is stochastic! The ...
0
votes
0answers
25 views

how to calculate number of frames in epsilon greedy with decay rate?

If starting epsilon is alpha and end epsilon is beta in epsilon greedy algorithm. discount rate is gamma and epsilon decay is lambda, how to calculate the F: number of frames to reach from alpha to ...
0
votes
0answers
16 views

Comparing the performance (reward) of dqn and logistic q-learning?

I have tried to compare my DQN results (rewards) with logistci q-learning (omitting the hidden layer, just inputs and outputs with a sigmoid activation function) My rewards of logistic-Q-N is about 5-...
0
votes
0answers
41 views

What are the reasons of select a optimizer to be SGD or Adam in DQN?Why?

I saw several comparison between SGD, RMSPROP and ADAM but what I am looking for is their comparsion in DQN algorithm? What is best to select as optimizer SGD or Adam in DQN? Why? Please check the ...
2
votes
1answer
42 views

“Each agent was evaluated every 250,000 training frames for 135,000 validation frames” What does this sentences stands for? in DQN nature paper?

In nature paper of DQN by DeepMind, DQN is compared to linear function but they does not said what is this linear function? They compared with some linear functions? 0- What is the meaning of this ...
3
votes
1answer
94 views

How to implement clipping the reward in DQN in keras

How to implement clipping the reward in DQN in keras? especially how to implement clipping the reward? Is this pseudo code correct: ...
3
votes
1answer
56 views

What are the effects of clipping the reward in stability?

I am looking for stabilizing my results of DQN, I found clipping is one technique to do it but I did not understand it completely! 1- what are the effects of clipping the reward, clipping the ...
1
vote
1answer
43 views

What is difference between final episodes of training and test in DQN?

What is difference between running in final episode of training mode and running in test mode in DQN? Is there any difference more than after training and tune the hyper-parameters, we test for one ...
1
vote
0answers
21 views

have deep reinforcement learning algorithms the unified behavior?

Could we say the behavior of different deep reinforcement learning algorithms is very similar for MDP and POMDP? Could we say DRL algorithms present a unified approach for finite-horizon, infinite ...
1
vote
0answers
29 views

DQN cannot learn or converge

I have implemented a DQN using keras. The task is to collect the circles and avoid the red circle and crosses. The associated rewards are +5, -5 and 0 otherwise. if the agent go out of the board, the ...
1
vote
1answer
215 views

what is difference between the DDQN and DQN?

I think I did not understand what is the difference between DQN and DDQN in implementation. I understand that we change the traget network during the running of DDQN but I do not understand how it is ...
0
votes
0answers
50 views

how to speed up running the DQN with the experience replay?

how to speed up running the DQN with the experience replay? Playing with which hyper parameters have the more effect on it? Because my dqn with experience replay is too slow!
2
votes
2answers
26 views

which of stable training results or good test results is more important?

which of stable training results or good test results are more important? For instance obtaining an unstable training accuracy in different episodes but good test accuracy is better or obtaining a ...
2
votes
0answers
47 views

How to resolve the instability of average reward per episode in training of DQN (Deep Q-Network)?

what is shown when average reward per episode in training is unstable? If there is big difference between average reward per episode and final reward by test section, what we can say? For ...
2
votes
1answer
41 views

Why does exploration in DQN not lead to instability?

Why does action exploration in DQN not lead to instability? I see in DQN algorithms, that it selects random actions even after some iterations. My question is how does this approach not lead to ...
2
votes
1answer
33 views

How we can have RF-QLearning or SVR-QLearning (Combine these algorithm with a Q-Learning )

How we can have RF-QLearning or SVR-QLearning (Combine these algorithm with a Q-Learning )? I want to replace the DNN section of Qlearning with a RF or SVR but the problem is that there is no clear ...
1
vote
1answer
28 views

Is the neural network in DQN used to learn like a supervised model?

Is the neural network in DQN used to learn like a supervised model?
1
vote
0answers
140 views

Why is my loss function for DQN converging too quickly?

I'm still relatively new to deep learning and am experiencing an issue that I can't seem to find a solution/explanation for. I've developed a DQN model in tensorflow, as described by DeepMind, and am ...
2
votes
1answer
54 views

What is wrong with this reinforcement learning environment ?

I'm working on below reinforcement learning problem: I have bottle of fix capacity (say 5 liters). At the bottom of bottle there is cock to remove water. The distribution of removal of water is not ...
1
vote
1answer
59 views

How to give rewards to actions in RL?

I'm working on below reinforcement learning problem: I have bottle of fix capacity (say 5 liters). At the bottom of bottle there is cock to remove water. The distribution of removal of water is not ...
1
vote
0answers
36 views

Deep Q-Learning with large number of actions

I'm using DQN with large number of actions in [0, 10000, step = 1000]. This means I have an action space of size 11 (including 0 and 10000). Action space is still discrete. My problem is that, instead ...
2
votes
1answer
85 views

Difference between advantages of Experience Replay in DQN2013 paper

I've been re-reading the Playing Atari with Deep Reinforcement Learning (2013) paper. It lists three advantages of experience replay: This approach has several advantages over standard online Q-...
0
votes
0answers
92 views

How to fold importance sampling weights into Huber Loss in a Deep-Q Network

I'm trying to implement Prioritized Experience Replay (PER) for a Deep-Q Network (DQN) as described in this paper: https://arxiv.org/pdf/1511.05952.pdf. In section 3.4 it discusses how priority ...
1
vote
0answers
76 views

Why is Distributional DQN faster than vanilla DQN?

Recently I learned about Distributional approach to RL, which is a quite fascinating and break--through algorithm. I have 2 questions: What is it that makes it perform so much better during runtime ...
1
vote
0answers
23 views

Can a DQN output a float result? [closed]

I'm a newbie of Deep Q Learning. After read some papers and tutorials on the web, I tried to train a DQN to control a game using TensorFlow. The input is the screenshoot of the game. The output is an ...
1
vote
0answers
96 views

Experience Replay Explain

I have read many blog articles, research papers and watched many youtube videos, but it seems it is hard to find why experience replay is efficient. I know that the experience replay stores (state, ...
1
vote
1answer
74 views

Reinforcement Learning with static state

Can Q Learning work with a static state for each step? What I mean by that is that the actions do not influence the following state at all. The episodes just iterate over the same data over and over ...
3
votes
1answer
620 views

Why random sample from replay for DQN?

I'm trying to gain an intuitive understanding of deep reinforcement learning. In deep Q-networks (DQN) we store all actions/environments/rewards in a memory array and at the end of the episode, "...
3
votes
1answer
83 views

Clamping Q function to it's theoretical maximum, yes or no?

I'm implementing DQN algorithm from scratch on MountainCar simulation. I'm using a setup of $reward = 1.0$ when car hits the flag, and $0$ otherwise. Reward decay factor is set to $\gamma=0.99$. ...
1
vote
1answer
738 views

What is a minimal setup to solve the CartPole-v0 with DQN?

I solved the CartPole-v0 with a CEM agent pretty easily (experiments and code), but I struggle to find a setup which works with DQN. Do you know which parameters should be adjusted so that the mean ...