Questions tagged [dqn]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

Reinforcement Learning - DQN - Training , Testing Challenges

I am trying to build a Reinforcement Learning - DQN model for hardware capacity optimization. The inputs to DQN model will be Cpu and Memory capacities along with the action size. The model predicts ...
3
votes
2answers
311 views

How exactly does DQN learn?

I created my custom environment in gym, which is a maze. I use a DQN model with ...
0
votes
0answers
10 views

Multi Agent Reinforcement Learning

I am trying to build multi agent reinforcement learning in 2D grid environment(Combat & Switch) using OpenAI. When I am trying to build game using ten agents, five opponents and the dimension is(...
0
votes
0answers
19 views

Epochs and other hyperparameters in Deep Q-Networks

I was wondering about hyperparameters used in Deep Q-Networks. Considering the use of replay memory and target network, together with the epsilon-greedy policy, are the number of epochs different of 1 ...
0
votes
1answer
391 views

DQN with decaying epsilon

I'm new to reinforcement learning. I'm studying DQN with decaying epsilon. I came across such example: EPISODES = 91 GAMMA = 0.2 EPSILON_DECAY = 0.999 MIN_EPSILON = 0.01 MAX_EPSILON = 1 My questions ...
0
votes
0answers
26 views

My Deep Q-Learning Network does not learn for OpenAI gym's cartpole problem

I am implementing OpenAI gym's cartpole problem using Deep Q-Learning (DQN). I followed tutorials (video and otherwise) and learned all about it. I implemented a code for myself and I thought it ...
1
vote
0answers
35 views

In a double Deep Q network what would happen if we switch the roles of both networks

We normally use the online network for action selection and the target network for evaluation , would there be a difference if we switched the roles? Because in the case Of Double Q learning, we ...
1
vote
0answers
42 views

Prioritized Experience Replay - which version is correct?

After reading a lot of stuff, I'm still not sure how to calculate the priorities for Prioritized Experience Replay (PER). Example code taken from here ...
0
votes
1answer
72 views

Is this a valid stability concern/improvement for DQN/DDQN reinforcement training?

As you all know, DQN or DDQN are known for "unstable training". Let's use the well known "CartPole". The agent has to balance the stick and gets a reward of +1 per frame. You can reach the 195 ...
1
vote
0answers
41 views

how do deep Q network deal with varying input size?

I am conducting research with multiply agents in an environment. The main concept of my methodology is a centralized control system, which means we take the positions, as well as other information, of ...
0
votes
1answer
32 views

Reinforcement (Q) learning: does it learn while in production?

I have a question for which I could not find the answer to it: While training reinforcement learning (using DQN), I get a model for the best reward for the next action. Now, if I deploy this model (i....
1
vote
1answer
28 views

How do I build a DQN which selects the correct objects in an environment based on the environment state?

I have an environment with 4 objects in it. All of these objects can either be selected or not selected. So the actions taken by my DQN should look like - ...
0
votes
0answers
32 views

Dueling DQN gradient with respect to a fully connected layer

Pertaining to this post, Dueling Network gradient with respect to Advantage stream, if Advantage and Value stream both obtain their values from a layer supposedly called A, such that Advantage = alpha....
1
vote
0answers
42 views

Can be possible to solve Rubik's cube using DQN?

I'm trying to solve Rubik's cube using deep learning and I came across with DQN, so I decided to give it a try. I developed all the code and started training bu I got this results: Loss goes up and ...
1
vote
2answers
32 views

Free a bit of RAM space

Here is Colab Notebook After 1500 episodes if batch_size=256, the RAM crashed. With Colab, I have the equivalent of 25.5 gigs of RAM. Is it normal? Or I don't have ...
0
votes
1answer
54 views

TF2.0 DQN has a loss of 0?

I am having a hard time understanding why my loss is constantly a zero when using DQN. I'm trying to use the gym environment to play the game ...
2
votes
1answer
101 views

Representation of state space, action space and reward system for RL porblem

I am trying to solve the problem of an agent dynamically discovering(start with no information about the environment) the environment and to explore as much of the environment as possible without ...
0
votes
0answers
200 views

Index tensor must have same dimensions as input tensor

I am trying to train a DQN to do optimal energy scheduling. Each state comes as a vector of 4 variables (represented by floats) saved in the replay memory as a state tensor, each action is an integer ...
1
vote
0answers
92 views

Q-learning, state transition, immediate rewards (trading logic)

I've been thinking about how to correctly calculate rewards for several weeks now. Here is a grid example: ...
0
votes
1answer
101 views

When should the last action be included in the state in reinforcement learning?

I am having some confusion as to whether the action should be included as part of the state input to an agent in a reinforcement learning setting (state-action pair). As from my observation, this is ...
2
votes
1answer
201 views

Would Deep Q Learning work for a finite horizon problem?

I want to apply Deep Q Learning to a problem, which has a clear finite horizon definition, like: $$V(s) = \mathbb{E}[r_1 + r_2]$$ Since the horizon is finite, I do not use reward discounting. My ...
1
vote
1answer
65 views

Having a reward structure which gives high positive rewards compared to the negative rewards

I am training an RL agent using PPO algorithm for a control problem. The objective of the agent is to maintain temperature in a room. It is an episodic task with episode length of 9 hrs and step size(...
0
votes
1answer
45 views

How to handle differences between training and deploying of an RL agent

Hi I am training an RL agent for a control problem. The objective of the agent is to maintain temperature in a zone. It is an episodic task with episode length of 10 hrs and actions being taken every ...
0
votes
1answer
175 views

Different results every time I train a reinforcement learning agent

I am training an RL agent for a control problem using PPO algorithm. I am using stable-baselines library for it. The objective of an agent is to maintain a temperature of 24 deg in a zone and it ...
3
votes
1answer
2k views

Evaluating a trained Reinforcement Learning Agent?

I am new to reinforcement learning agent training. I have read about PPO algorithm and used stable baselines library to train an agent using PPO. So my question here is how do I evaluate a trained RL ...
0
votes
1answer
1k views

Policy Gradient with continuous action space

How to apply reinforce/policy-gradient algorithms for continuous action space. I have learnt that one of the advantages of policy gradients is , it is applicable for continuous action space. One way I ...
2
votes
1answer
214 views

Deep reinforcement learning on changing data sizes

I have a game that I want to build a model that will learn to play the game. Yet, the environment output is two lists that represent the location and number of soldiers of the user and Opponent. The ...
1
vote
2answers
1k views

Agent always takes a same action in DQN - Reinforcement Learning

I have trained an RL agent using DQN algorithm. After 20000 episodes my rewards are converged. Now when I test this agent, the agent is always taking the same action , irrespective of state. I find ...
0
votes
2answers
266 views

How to calculate Temperature variable in softmax(boltzmann) exploration

Hi I am developing a reinforcement learning agent for a continous state/discrete action space. I am trying to use boltmzann/softmax exploration as action selection strategy. My action space is of size ...
1
vote
0answers
148 views

Reducing the training time of an RL agent

I am trying to develop an rl agent using DQN algorithm.During training, the agent interacts with environment which is a simulated one.Each episode takes around 10 mins to run. This way if want my ...
1
vote
1answer
86 views

How to formulate reward of an rl agent with two objectives

I have started learning reinforcement learning and trying to apply it for my use case. I am developing an rl agent which can maintain temperature at a particular value, and minimize the energy ...
1
vote
1answer
32 views

Is DQN limited to working with only image frames?

I have few questions about Deep Q Network. Does DQN only accept image frames as input? I have never hear (read) a paperwork where it doesn't use image frames. If the first is a No, then does image ...
1
vote
0answers
56 views

How can I improve the performance of my DQN?

I created a deep Q network to play snake. The code works fine, except for the fact that performance doesn't really improve over the training cycle. At the end, it's pretty much indistinguishable from ...
0
votes
1answer
60 views

Deep Q-Learning for physical quantity: q-values distribution not as expected

Setting I am trying to learn a specific physical quantity (radiance) inside a 3D scene with Deep Q-Learning. Just to give a quick overview, my agent shoots rays inside the scene: the reward is the ...
1
vote
0answers
46 views

DQNs for huge or continuous state spaces [closed]

Have there been occasions where DQN failed to deal with huge state spaces? Can you point out a research paper regarding it?
2
votes
1answer
754 views

Difference between Dueling DQN and Double DQN?

I have read some articles, but still can not figure out the difference between the Dueling DQN and Double DQN? What exactly is the difference between them? Also, Does Dueling DQN need to be built on ...
0
votes
0answers
275 views

How to train DDQN model in reinforcement learning?

I was reading through some RL code but couldn't understand one small bit. here is the code: ...
0
votes
1answer
284 views

How does DQN solve Open AI Cartpole - v0?

Context I am confused about how a DQN is supposed to solve the cart pole problem since the rewards are so dense. I have been using pytorch example. I am aware of some solutions, but I have issue with ...
0
votes
1answer
981 views

Q-Learning experience replay: how to feed the neural network?

I'm trying to replicate the DQN Atari experiment. Actually my DQN isn't performing well; checking another one's codes, I saw something about experience replay which I don't understand. First, when you ...
3
votes
1answer
663 views

DQN fails to find optimal policy

Based on DeepMind publication, I've recreated the environment and I am trying to make the DQN find and converge to an optimal policy. The task of an agent is to learn how to sustainably collect apples ...
1
vote
0answers
24 views

Is reward accumulated during a play iteration when performing SARSA?

I've been having issue with getting my DQN to converge to a good solution for snake. Regardless of the different types of reward functions I've tried, it seems that the snake is indefinitely going ...
3
votes
0answers
650 views

Deep Reinforcement Learning for dynamic pricing

I am trying to implement a Deep Q Network model for Dynamic pricing in Logistics. I can define State Space (Origin, Destination, type of the shipment, customer, Type of the product, Commodity of the ...
6
votes
2answers
643 views

How to choose between discounted reward and average reward?

How to select between average reward and discounted reward? And when average reward is more effective in comparison with discounter reward and when vice versa is correct? Is is possible to use both ...
1
vote
0answers
32 views

What is the meaning of the Variant Q-learning and To what INPUT and OUTPUT refer? in Abstract of DeepMind DQN paper 2013

-INPUT and OUTPUT OF ATARI DQN: In the abstract paragraph of the DQN work by DeepMind it has written: " We present the first deep learning model to successfully learn control policies directly ...
0
votes
1answer
2k views

RL - Weighthing negative rewards [closed]

Let's consider that I give an agent a reward of -1 (minimum reward) every time it performs an action which leads to the premature end of the episode (i.e., the agent dies). Besides, I also give a ...
8
votes
2answers
2k views

How does Implicit Quantile-Regression Network (IQN) differ from QR-DQN?

For several months I browsed the internet hoping to find a user-friendly explanation of the Implicit Quantile Regression Network (IQN). But, it seems there is none at all. How does IQN differ from ...
1
vote
1answer
67 views

Will reinforcement learning work if states wont get repeated again?

I am working on a information retrieval model where the user enters a query and the model has to retrieve 3 most relevant FAQ pairs.I am collecting implicit feedback in terms of page clicks etc.What I ...
1
vote
0answers
183 views

comparison of linear Q-learning and DQN

I saw in DQN nature paper 2015 https://www.nature.com/articles/nature14236(Extended Data Table 4) some comparisons between DQN and linear Q-learning. The ratio ...
0
votes
1answer
351 views

is it acceptable if the reward of test of DQN is lower than reward of training of DQN in minimization problem?

if we train a DQN over 40000-60000 episodes for 500 time steps. The mean of reward during last 100 training steps is about 1.1 times of reward during the test process. Environment is stochastic! The ...
2
votes
1answer
52 views

“Each agent was evaluated every 250,000 training frames for 135,000 validation frames” What does this sentences stands for? in DQN nature paper?

In nature paper of DQN by DeepMind, DQN is compared to linear function but they does not said what is this linear function? They compared with some linear functions? 0- What is the meaning of this ...