Skip to main content

Questions tagged [dqn]

The tag has no usage guidance, but it has a tag wiki.

Filter by
Sorted by
Tagged with
1 vote
0 answers
31 views

Is reward accumulated during a play iteration when performing SARSA?

I've been having issue with getting my DQN to converge to a good solution for snake. Regardless of the different types of reward functions I've tried, it seems that the snake is indefinitely going ...
DevarakondaV's user avatar
3 votes
0 answers
725 views

Deep Reinforcement Learning for dynamic pricing

I am trying to implement a Deep Q Network model for Dynamic pricing in Logistics. I can define State Space (Origin, Destination, type of the shipment, customer, Type of the product, Commodity of the ...
Karthik Rajkumar's user avatar
6 votes
2 answers
2k views

How to choose between discounted reward and average reward?

How to select between average reward and discounted reward? And when average reward is more effective in comparison with discounter reward and when vice versa is correct? Is is possible to use both ...
user10296606's user avatar
  • 1,844
1 vote
0 answers
37 views

What is the meaning of the Variant Q-learning and To what INPUT and OUTPUT refer? in Abstract of DeepMind DQN paper 2013

-INPUT and OUTPUT OF ATARI DQN: In the abstract paragraph of the DQN work by DeepMind it has written: " We present the first deep learning model to successfully learn control policies directly ...
user10296606's user avatar
  • 1,844
1 vote
1 answer
2k views

RL - Weighthing negative rewards [closed]

Let's consider that I give an agent a reward of -1 (minimum reward) every time it performs an action which leads to the premature end of the episode (i.e., the agent dies). Besides, I also give a ...
nil's user avatar
  • 88
11 votes
2 answers
3k views

How does Implicit Quantile-Regression Network (IQN) differ from QR-DQN?

For several months I browsed the internet hoping to find a user-friendly explanation of the Implicit Quantile Regression Network (IQN). But, it seems there is none at all. How does IQN differ from ...
Kari's user avatar
  • 2,736
1 vote
1 answer
263 views

Will reinforcement learning work if states wont get repeated again?

I am working on a information retrieval model where the user enters a query and the model has to retrieve 3 most relevant FAQ pairs.I am collecting implicit feedback in terms of page clicks etc.What I ...
Charan Reddy's user avatar
1 vote
0 answers
310 views

comparison of linear Q-learning and DQN

I saw in DQN nature paper 2015 https://www.nature.com/articles/nature14236(Extended Data Table 4) some comparisons between DQN and linear Q-learning. The ratio ...
user10296606's user avatar
  • 1,844
0 votes
1 answer
675 views

is it acceptable if the reward of test of DQN is lower than reward of training of DQN in minimization problem?

if we train a DQN over 40000-60000 episodes for 500 time steps. The mean of reward during last 100 training steps is about 1.1 times of reward during the test process. Environment is stochastic! The ...
user10296606's user avatar
  • 1,844
2 votes
1 answer
66 views

"Each agent was evaluated every 250,000 training frames for 135,000 validation frames" What does this sentences stands for? in DQN nature paper?

In nature paper of DQN by DeepMind, DQN is compared to linear function but they does not said what is this linear function? They compared with some linear functions? 0- What is the meaning of this ...
user10296606's user avatar
  • 1,844
4 votes
1 answer
648 views

How to implement clipping the reward in DQN in keras

How to implement clipping the reward in DQN in keras? especially how to implement clipping the reward? Is this pseudo code correct: ...
user10296606's user avatar
  • 1,844
7 votes
1 answer
4k views

What are the effects of clipping the reward in stability?

I am looking for stabilizing my results of DQN, I found clipping is one technique to do it but I did not understand it completely! 1- what are the effects of clipping the reward, clipping the ...
user10296606's user avatar
  • 1,844
1 vote
1 answer
453 views

What is difference between final episodes of training and test in DQN?

What is difference between running in final episode of training mode and running in test mode in DQN? Is there any difference more than after training and tune the hyper-parameters, we test for one ...
user10296606's user avatar
  • 1,844
1 vote
0 answers
1k views

DQN cannot learn or converge

I have implemented a DQN using keras. The task is to collect the circles and avoid the red circle and crosses. The associated rewards are +5, -5 and 0 otherwise. if the agent go out of the board, the ...
user avatar
10 votes
2 answers
18k views

what is difference between the DDQN and DQN?

I think I did not understand what is the difference between DQN and DDQN in implementation. I understand that we change the traget network during the running of DDQN but I do not understand how it is ...
user10296606's user avatar
  • 1,844
2 votes
3 answers
346 views

Which is more important - stable training results or good test results?

Which is more important - stable training results or good test results? For instance, is obtaining an unstable training accuracy in different epochs, but good test accuracy better? Or is obtaining a ...
user10296606's user avatar
  • 1,844
2 votes
0 answers
225 views

How to resolve the instability of average reward per episode in training of DQN (Deep Q-Network)?

what is shown when average reward per episode in training is unstable? If there is big difference between average reward per episode and final reward by test section, what we can say? For instance in ...
user10296606's user avatar
  • 1,844
4 votes
1 answer
2k views

Why does exploration in DQN not lead to instability?

Why does action exploration in DQN not lead to instability? I see in DQN algorithms, that it selects random actions even after some iterations. My question is how does this approach not lead to ...
user10296606's user avatar
  • 1,844
2 votes
1 answer
136 views

How we can have RF-QLearning or SVR-QLearning (Combine these algorithm with a Q-Learning )

How we can have RF-QLearning or SVR-QLearning (Combine these algorithm with a Q-Learning )? I want to replace the DNN section of Qlearning with a RF or SVR but the problem is that there is no clear ...
user10296606's user avatar
  • 1,844
2 votes
1 answer
634 views

Is the neural network in DQN used to learn like a supervised model?

Is the neural network in DQN used to learn like a supervised model?
user10296606's user avatar
  • 1,844
1 vote
0 answers
1k views

Why is my loss function for DQN converging too quickly?

I'm still relatively new to deep learning and am experiencing an issue that I can't seem to find a solution/explanation for. I've developed a DQN model in tensorflow, as described by DeepMind, and am ...
DevarakondaV's user avatar
3 votes
1 answer
875 views

What is wrong with this reinforcement learning environment ?

I'm working on below reinforcement learning problem: I have bottle of fix capacity (say 5 liters). At the bottom of bottle there is cock to remove water. The distribution of removal of water is not ...
Krishna Nevase's user avatar
2 votes
1 answer
599 views

How to give rewards to actions in RL?

I'm working on below reinforcement learning problem: I have bottle of fix capacity (say 5 liters). At the bottom of bottle there is cock to remove water. The distribution of removal of water is not ...
Krishna Nevase's user avatar
1 vote
0 answers
746 views

Deep Q-Learning with large number of actions

I'm using DQN with large number of actions in [0, 10000, step = 1000]. This means I have an action space of size 11 (including 0 and 10000). Action space is still discrete. My problem is that, instead ...
Krishna Nevase's user avatar
5 votes
1 answer
1k views

Difference between advantages of Experience Replay in DQN2013 paper

I've been re-reading the Playing Atari with Deep Reinforcement Learning (2013) paper. It lists three advantages of experience replay: This approach has several advantages over standard online Q-...
seungjaeryanlee's user avatar
2 votes
1 answer
1k views

Why is Distributional DQN faster than vanilla DQN?

Recently I learned about Distributional approach to RL, which is a quite fascinating and break--through algorithm. I have 2 questions: What is it that makes it perform so much better during runtime ...
Kari's user avatar
  • 2,736
1 vote
0 answers
118 views

Can a DQN output a float result? [closed]

I'm a newbie of Deep Q Learning. After read some papers and tutorials on the web, I tried to train a DQN to control a game using TensorFlow. The input is the screenshoot of the game. The output is an ...
shell's user avatar
  • 11
1 vote
0 answers
182 views

Experience Replay Explain

I have read many blog articles, research papers and watched many youtube videos, but it seems it is hard to find why experience replay is efficient. I know that the experience replay stores (state, ...
Dongyu Kang's user avatar
1 vote
1 answer
462 views

Reinforcement Learning with static state

Can Q Learning work with a static state for each step? What I mean by that is that the actions do not influence the following state at all. The episodes just iterate over the same data over and over ...
Pit Wegner's user avatar
7 votes
3 answers
4k views

Why random sample from replay for DQN?

I'm trying to gain an intuitive understanding of deep reinforcement learning. In deep Q-networks (DQN) we store all actions/environments/rewards in a memory array and at the end of the episode, "...
ZAR's user avatar
  • 203
3 votes
1 answer
363 views

Clamping Q function to it's theoretical maximum, yes or no?

I'm implementing DQN algorithm from scratch on MountainCar simulation. I'm using a setup of $reward = 1.0$ when car hits the flag, and $0$ otherwise. Reward decay factor is set to $\gamma=0.99$. ...
omittones's user avatar
  • 133
4 votes
1 answer
2k views

What is a minimal setup to solve the CartPole-v0 with DQN?

I solved the CartPole-v0 with a CEM agent pretty easily (experiments and code), but I struggle to find a setup which works with DQN. Do you know which parameters should be adjusted so that the mean ...
Martin Thoma's user avatar

1
2