Questions tagged [dqn]
85
questions
0
votes
0
answers
23
views
Feed two matrices into an mlp neural network
So Im trying to train a recommender system (w/ DQN) using two sets of data , first is a 2D array size $N\times N$ where the diagonal is the current content (= state) and the rest row is the ...
0
votes
0
answers
18
views
Dueling DQN with varying number of actions
I have an RL problem, where the number of actions depends on the state. Furthermore, each action-value computation requires action information in the form of a high-dimensional, continuous vector in ...
0
votes
0
answers
24
views
State and next state is the same on openai gym atari
i am using OpenGym atari to trainning my Pacman agent, this is part of my code
...
- 1
0
votes
0
answers
27
views
Problem of extreamly varied reward value in DDQN
I am trying to train my model by DDQN agent after creating a customized environment in gym. I am stating my hyper-parameters and other details here.
state shape = 5
action space = 0,1,2, ..., 100
...
- 103
0
votes
0
answers
125
views
Reinforcement Learning (gymnasium's FrozenLake-v1) using Spiking Neural Networks (BindsNet)
I'm new to reinforcement learning. I'm trying to solve the FrozenLake-v1 game using OpenAI's gymnasium learning environment and BindsNet, which is a library to simulate Spiking Neural Networks using ...
- 1
0
votes
0
answers
9
views
How does one decide what should be the state in deep q learning
Can anyone share the thought process of how the state is formed in Deep Q learning ? I am not indicating to the basic examples of OpenAI gym but more practical usages of DQN like Job scheduling , etc.
...
- 1
0
votes
0
answers
19
views
How to decide a State for Deep Q Learning for Production Line scheduling
There is a production floor with W workstations and N jobs with M operations( different processing times per operation ). A job is completed only if its M Operations are completed. Objective is to ...
- 1
0
votes
2
answers
122
views
In a convolutional layer, is it standard practice to modify stride and padding to get a desired output?
I'm trying to implement the CNN described in A Framework of Hierarchical Deep Q-Network for Portfolio Management (see screenshot). In the paper, the author describes the first CNN layer as having a ...
2
votes
1
answer
128
views
How to Form the Training Examples for Deep Q Network in Reinforcement Learning?
Trying to pick up basics of reinforcement learning by self-study from some blogs and texts. Forgive me if the question is too basic and different bits that I understand are a bit messy, but even after ...
- 315
0
votes
1
answer
871
views
gym car racing v0 using DQN
I am currently learning reinforcement learning and wanted to use it on the car racing-v0 environment. I have successfully made it using PPO algorithm and now I want to use a DQN algorithm but when I ...
- 1
1
vote
1
answer
57
views
Can Online DQN model overfit?
I am new in the area of RL and currently trying to train an online DQN model. Can an online model overfit since its always learning? and how can I tell if that happens?
- 11
1
vote
1
answer
161
views
Cartpole - Number of layers and neurons - model hyperparameters
Can anyone please suggest me how to arrive to the best optimal values for number of layers, number of neurons parameters of the deep learning model in DDQN algorithm for cartpole problem.
As input and ...
- 21
1
vote
1
answer
162
views
In DQN, why not use target network to predict current state Q values?
In DQN, why not use target network to predict current state Q values, and not only next state q values?
In doing a basic dq learning algorithm with nn from scratch, with replay memory, and minibatch ...
1
vote
0
answers
187
views
DQN CartPole-v1 neural network doesn't optimize
I'm doing my first dnq algorithm, I'm trying to build a dnq agent, and neural network from scratch, but it seems that neural network doesn't optimize, I did 2 hidden layers, with ReLU, and the output ...
0
votes
0
answers
86
views
Q values loss per episode and mean absolute error
I am new to deep reinforcement learning! I am following this code for my adaptation problem (doing actions)
https://github.com/jaromiru/AI-blog/blob/master/CartPole-DQN.py
I am wondering how I can ...
2
votes
1
answer
136
views
Dimensionality of the target for DQN agent training
From what I understand, a DQN agent has as many outputs as there are actions (for each state). If we consider a scalar state with 4 actions, that would mean that the DQN would have a 4 dimensional ...
- 21
1
vote
0
answers
238
views
DQN DDQN and 3DQN differences?
I'm doing a course on reinforcement learning, and one of our tasks is to implement an agent on the Lunar lander continuous V2 environment from openAI gym.
In order to solve the continuous problem, I ...
- 11
1
vote
0
answers
11
views
Implementing DQN - Pylessons.com article - Queries [closed]
I have followed the below link to implement DQN Algorithm
https://pylessons.com/CartPole-reinforcement-learning/
Can someone explain me:
Why do we need to have else condition in act function during ...
- 21
3
votes
2
answers
485
views
How exactly does DQN learn?
I created my custom environment in gym, which is a maze. I use a DQN model with ...
- 31
0
votes
0
answers
31
views
Epochs and other hyperparameters in Deep Q-Networks
I was wondering about hyperparameters used in Deep Q-Networks. Considering the use of replay memory and target network, together with the epsilon-greedy policy, are the number of epochs different of 1 ...
- 13
2
votes
1
answer
4k
views
DQN with decaying epsilon
I'm new to reinforcement learning. I'm studying DQN with decaying epsilon. I came across such example:
EPISODES = 91
GAMMA = 0.2
EPSILON_DECAY = 0.999
MIN_EPSILON = 0.01
MAX_EPSILON = 1
My questions ...
- 123
1
vote
0
answers
46
views
In a double Deep Q network what would happen if we switch the roles of both networks
We normally use the online network for action selection and the target network for evaluation , would there be a difference if we switched the roles?
Because in the case Of Double Q learning, we ...
- 11
1
vote
0
answers
146
views
Prioritized Experience Replay - which version is correct?
After reading a lot of stuff, I'm still not sure how to calculate the priorities for Prioritized Experience Replay (PER).
Example code taken from here
...
- 13
0
votes
1
answer
300
views
Is this a valid stability concern/improvement for DQN/DDQN reinforcement training?
As you all know, DQN or DDQN are known for "unstable training".
Let's use the well known "CartPole". The agent has to balance the stick and gets a reward of +1 per frame. You can reach the 195 ...
- 13
1
vote
0
answers
215
views
how do deep Q network deal with varying input size?
I am conducting research with multiply agents in an environment. The main concept of my methodology is a centralized control system, which means we take the positions, as well as other information, of ...
- 11
0
votes
1
answer
66
views
Reinforcement (Q) learning: does it learn while in production?
I have a question for which I could not find the answer to it:
While training reinforcement learning (using DQN), I get a model for the best reward for the next action. Now, if I deploy this model (i....
- 103
1
vote
1
answer
173
views
How do I build a DQN which selects the correct objects in an environment based on the environment state?
I have an environment with 4 objects in it. All of these objects can either be selected or not selected. So the actions taken by my DQN should look like - ...
3
votes
1
answer
197
views
Is it possible to solve Rubik's cube using DQN?
I'm trying to solve Rubik's cube using deep learning and I came across with DQN, so I decided to give it a try. I developed all the code and started training but I got this results:
Loss goes up and ...
1
vote
2
answers
75
views
Free a bit of RAM space
Here is Colab Notebook
After 1500 episodes if batch_size=256, the RAM crashed. With Colab, I have the equivalent of 25.5 gigs of RAM. Is it normal? Or I don't have ...
- 131
1
vote
1
answer
118
views
TF2.0 DQN has a loss of 0?
I am having a hard time understanding why my loss is constantly a zero when using DQN.
I'm trying to use the gym environment to play the game ...
- 111
2
votes
1
answer
319
views
Representation of state space, action space and reward system for RL porblem
I am trying to solve the problem of an agent dynamically discovering(start with no information about the environment) the environment and to explore as much of the environment as possible without ...
1
vote
0
answers
116
views
Q-learning, state transition, immediate rewards (trading logic)
I've been thinking about how to correctly calculate rewards for several weeks now.
Here is a grid example:
...
- 21
2
votes
1
answer
1k
views
When should the last action be included in the state in reinforcement learning?
I am having some confusion as to whether the action should be included as part of the state input to an agent in a reinforcement learning setting (state-action pair). As from my observation, this is ...
- 21
2
votes
1
answer
752
views
Would Deep Q Learning work for a finite horizon problem?
I want to apply Deep Q Learning to a problem, which has a clear finite horizon definition, like:
$$V(s) = \mathbb{E}[r_1 + r_2]$$
Since the horizon is finite, I do not use reward discounting. My ...
- 203
2
votes
1
answer
464
views
Having a reward structure which gives high positive rewards compared to the negative rewards
I am training an RL agent using PPO algorithm for a control problem. The objective of the agent is to maintain temperature in a room. It is an episodic task with episode length of 9 hrs and step size(...
- 555
0
votes
1
answer
166
views
How to handle differences between training and deploying of an RL agent
Hi I am training an RL agent for a control problem. The objective of the agent is to maintain temperature in a zone. It is an episodic task with episode length of 10 hrs and actions being taken every ...
- 555
1
vote
1
answer
1k
views
Different results every time I train a reinforcement learning agent
I am training an RL agent for a control problem using PPO algorithm. I am using stable-baselines library for it.
The objective of an agent is to maintain a temperature of 24 deg in a zone and it ...
- 555
3
votes
1
answer
5k
views
Evaluating a trained Reinforcement Learning Agent?
I am new to reinforcement learning agent training. I have read about PPO algorithm and used stable baselines library to train an agent using PPO. So my question here is how do I evaluate a trained RL ...
- 555
2
votes
1
answer
5k
views
Policy Gradient with continuous action space
How to apply reinforce/policy-gradient algorithms for continuous action space. I have learnt that one of the advantages of policy gradients is , it is applicable for continuous action space. One way I ...
- 555
3
votes
1
answer
885
views
Deep reinforcement learning on changing data sizes
I have a game that I want to build a model that will learn to play the game. Yet, the environment output is two lists that represent the location and number of soldiers of the user and Opponent.
The ...
4
votes
2
answers
4k
views
Agent always takes a same action in DQN - Reinforcement Learning
I have trained an RL agent using DQN algorithm. After 20000 episodes my rewards are converged. Now when I test this agent, the agent is always taking the same action , irrespective of state. I find ...
- 555
1
vote
2
answers
697
views
How to calculate Temperature variable in softmax(boltzmann) exploration
Hi I am developing a reinforcement learning agent for a continous state/discrete action space. I am trying to use boltmzann/softmax exploration as action selection strategy. My action space is of size ...
- 555
1
vote
0
answers
418
views
Reducing the training time of an RL agent
I am trying to develop an rl agent using DQN algorithm.During training, the agent interacts with environment which is a simulated one.Each episode takes around 10 mins to run. This way if want my ...
- 555
1
vote
1
answer
191
views
How to formulate reward of an rl agent with two objectives
I have started learning reinforcement learning and trying to apply it for my use case. I am developing an rl agent which can maintain temperature at a particular value, and minimize the energy ...
- 555
1
vote
1
answer
131
views
Is DQN limited to working with only image frames?
I have few questions about Deep Q Network.
Does DQN only accept image frames as input? I have never hear (read) a paperwork where it doesn't use image frames.
If the first is a No, then does image ...
- 13
1
vote
0
answers
120
views
How can I improve the performance of my DQN?
I created a deep Q network to play snake. The code works fine, except for the fact that performance doesn't really improve over the training cycle. At the end, it's pretty much indistinguishable from ...
- 79
0
votes
1
answer
71
views
Deep Q-Learning for physical quantity: q-values distribution not as expected
Setting
I am trying to learn a specific physical quantity (radiance) inside a 3D scene with Deep Q-Learning. Just to give a quick overview, my agent shoots rays inside the scene: the reward is the ...
- 274
1
vote
0
answers
64
views
DQNs for huge or continuous state spaces [closed]
Have there been occasions where DQN failed to deal with huge state spaces? Can you point out a research paper regarding it?
- 11
3
votes
1
answer
2k
views
Difference between Dueling DQN and Double DQN?
I have read some articles, but still can not figure out the difference between the Dueling DQN and Double DQN? What exactly is the difference between them? Also, Does Dueling DQN need to be built on ...
- 2,725
0
votes
0
answers
488
views
How to train DDQN model in reinforcement learning?
I was reading through some RL code but couldn't understand one small bit.
here is the code:
...
- 101