Newest 'q-learning' Questions

0

votes

0answers

6 views

In reinforcement learning, continue update weights after training?

Assuming an rl agent that has to be trained, and then you can mainain same weights over episodes: in a single episode, you have to firtly backup weights, udpate continuosly weights in the episodes, ...

asked 2 days ago

Lorenzo Tinfena

194 bronze badges

-1

votes

0answers

16 views

Deep q learning from scratch weights diverge to NaN

I'm trying to make a deep q learning algorithm with neural network from scratch, minbatch gradient descent, replay memory, and target network. But weights diverge to NaN after a around 40 episodes ...

deep-learning neural-network reinforcement-learning q-learning dqn

asked May 7 at 16:21

Lorenzo Tinfena

194 bronze badges

1

vote

1answer

22 views

In DQN, why not use target network to predict current state Q values?

In DQN, why not use target network to predict current state Q values, and not only next state q values? In doing a basic dq learning algorithm with nn from scratch, with replay memory, and minibatch ...

machine-learning deep-learning reinforcement-learning q-learning dqn

asked May 7 at 11:46

Lorenzo Tinfena

194 bronze badges

0

votes

2answers

19 views

Zero Padding in Convolutional neural network

We use Convolutional neural network because it by design learns features that generalize over spatial location , so when using conv operation it reduce image size and that what we hope to have so we ...

deep-learning cnn q-learning

asked May 7 at 9:05

user117272

91 bronze badge

0

votes

0answers

15 views

Convolutional neural network with Deep Q learning in games

What does "the average magnitude of maximal action value output by the network" tell us? I mean if we plot this graph, is it good to start as low value and then increase until it goes in a ...

deep-learning convolutional-neural-network q-learning game

asked May 7 at 8:24

user117272

91 bronze badge

3

votes

1answer

27 views

Exploration in Q learning: Epsilon greedy vs Exploration function

I am trying to understand how to make sure that our agent explores the state space enough before exploiting what it knows. I am aware that we use epsilon-greedy approach with a decaying epsilon to ...

machine-learning-model q-learning

asked May 5 at 15:40

JANVI SHARMA

453 bronze badges

0

votes

0answers

14 views

Reinforcement Learning in continuing tasks (non-episodic)

I'm looking for an algorithm for Deep Reinforcement Learning in non-episodic or continuing tasks. To be explicit, I'm looking for an algorithm that allows the agent to learn online without separate ...

reinforcement-learning q-learning

asked May 1 at 20:54

Tomás Lara

113 bronze badges

1

vote

0answers

15 views

DQN CartPole-v1 neural network doesn't optimize

I'm doing my first dnq algorithm, I'm trying to build a dnq agent, and neural network from scratch, but it seems that neural network doesn't optimize, I did 2 hidden layers, with ReLU, and the output ...

deep-learning neural-network reinforcement-learning q-learning dqn

asked Apr 30 at 15:41

Lorenzo Tinfena

194 bronze badges

2

votes

0answers

24 views

“Learning” algorithm to use when future depends on past events (MDP property not met)

There are around 5 different retirement plans available in my country. People can pick from them freely. I would like to create a solution that would try to predict the best plan(s) given a particular ...

q-learning markov-process ai

asked Apr 30 at 8:36

White_Raven

1212 bronze badges

0

votes

0answers

14 views

How to use binary numbers and plain floats in the same neural network input for reinforcement learning?

I'm modeling a system whose configuration can be represented by a binary array ([1,0,0,1] or [0,1,0,0], for example), and the agent can move on a 2D space (thus having 3 DoFs), and the action the ...

tensorflow reinforcement-learning q-learning

asked Apr 13 at 19:38

Lucas Pelizzari

1

vote

1answer

30 views

On what principle did Google's DeepMind learn to walk?

I just saw this video on Youtube. On what principle did Google's DeepMind learn to walk? Was it Q-Learning or a Genetic Algorithm or Policy Gradient?

machine-learning deep-learning q-learning genetic-algorithms deepmind

asked Mar 29 at 11:45

learner

233 bronze badges

2

votes

1answer

29 views

Dimensionality of the target for DQN agent training

From what I understand, a DQN agent has as many outputs as there are actions (for each state). If we consider a scalar state with 4 actions, that would mean that the DQN would have a 4 dimensional ...

machine-learning deep-learning q-learning dqn

asked Mar 28 at 14:59

Dhoop

213 bronze badges

0

votes

0answers

9 views

What are the differences between DQN DDQN and Dueling Q network?

What are the differences between DQN DDQN and Dueling Q network? And how they expressed in the agent's behaviour? (For example in an Open AI Gym environment)

reinforcement-learning q-learning dqn

asked Mar 17 at 16:26

user113367

111 bronze badge

1

vote

0answers

26 views

Deep Q-learning in non-episodic tasks

I want to use Deep Q-learning (specifically DDQL by Hasselt et al. 2015, but it is the same principle) in a non-episodic task (continuing). I know that it is possible to use Q-learning in continuing ...

reinforcement-learning q-learning

asked Mar 15 at 21:38

Tomás Lara

113 bronze badges

1

vote

0answers

23 views

DQN DDQN and 3DQN differences?

I'm doing a course on reinforcement learning, and one of our tasks is to implement an agent on the Lunar lander continuous V2 environment from openAI gym. In order to solve the continuous problem, I ...

reinforcement-learning q-learning dqn

asked Mar 15 at 13:30

user113367

111 bronze badge

1

vote

0answers

7 views

Implementing DQN - Pylessons.com article - Queries [closed]

I have followed the below link to implement DQN Algorithm https://pylessons.com/CartPole-reinforcement-learning/ Can someone explain me: Why do we need to have else condition in act function during ...

reinforcement-learning q-learning dqn

asked Mar 11 at 11:50

vimala

111 bronze badge

2

votes

1answer

60 views

Does convergence equal learning in Deep Q-learning?

In my current research project I'm using the Deep Q-learning algorithm. The setup is as follows: I'm training the model (using Deep Q-learning) on a static dataset made up of experiences extracted ...

deep-learning reinforcement-learning q-learning convergence generalization

asked Feb 21 at 16:00

Aeryan

431 silver badge5 bronze badges

0

votes

0answers

11 views

How can my loss be stable while the gradient keeps growing?

I have been working on an Offline/Batch Reinforcement Learning problem where I am using a BCQ-DDQN model as a Q-table. The model input is a state of 8 dimensions, and the output is a vector of Q-...

deep-learning reinforcement-learning loss-function q-learning gradient

asked Jan 27 at 13:23

MetaHG

212 bronze badges

0

votes

1answer

36 views

How to report results of RL with high variance?

I run Q-learning and SARSA algortihms on the same problem but the results fluctuate heavily and when I draw them, there is no smooth graph. How should I repost the results? I run algorithms for 500 ...

reinforcement-learning q-learning

asked Jan 6 at 19:41

Katatonia

2012 silver badges8 bronze badges

0

votes

0answers

13 views

Tensorflow deep Q neural network wont learn with more than 1 hidden layer

I've been using TensorFlow in Python with a very simple deep Q neural network (NN) to play a very simple game. The game is in an array [0, 0, 0, 0] and by picking one of the positions 0, 1, 2, 3, the ...

machine-learning deep-learning q-learning

asked Dec 30 '20 at 18:32

Oliver Nicholls

11 bronze badge

2

votes

0answers

43 views

Offline/Batch Reinforcement Learning: Doubly Robust Off-policy Estimator takes huge values

Context: My team and I are working on a RL problem for a specific application. We have data collected from user interactions (states, actions, etc.). It is too costly for us to emulate agents. We ...

machine-learning dataset reinforcement-learning q-learning estimators

asked Dec 4 '20 at 14:41

MetaHG

212 bronze badges

0

votes

0answers

48 views

Cannot solve FrozenLake with QLearning and slippery mode enabled

I'm trying to solve the open-ai gym frozen lake problem with a basic Q learning table and it works quite well with is_slippery=False. But it cannot find a solution ...

python q-learning openai-gym

asked Nov 15 '20 at 18:01

Marc

1214 bronze badges

1

vote

1answer

70 views

Definition of the Q* function in reinforcement learning

I'm making my way through Sutton's Introduction to Reinforcement Learning. He gives the definition of the $q_*$ function as follows $$ q_*(a) = \mathbf{E}[R_t | A_t = a] $$ where $A_t$ is the action ...

machine-learning reinforcement-learning q-learning

asked Oct 10 '20 at 22:37

marlineer43

132 bronze badges

2

votes

0answers

21 views

Reference implementation of q-learning in Python

I'm a machine learning newbie, trying to learn Q-learning. I read a few texts and I get the general gist, but what I'd really love to see is a simple example of a Q-learning algorithm in Python that I ...

machine-learning python q-learning implementation

asked Sep 5 '20 at 16:15

Ram Rachum

1613 bronze badges

0

votes

1answer

32 views

Different Initial Q-Values in Q-Learning

When working with Q-Learning, what is the difference between having a Q_0(a) with all values zero, random or optimistic?

reinforcement-learning q-learning

asked Aug 30 '20 at 10:25

Giulia

1

0

votes

1answer

38 views

When should I use normal Q learning over a DQN?

From this article here, it says that using a tabular Q function is less scalable than a deep Q network. I assume that this means that the Q table approach works for some environments, but once they ...

reinforcement-learning q-learning

asked Aug 27 '20 at 0:09

jasooney23

31 bronze badge

0

votes

0answers

27 views

My Deep Q-Learning Network does not learn for OpenAI gym's cartpole problem

I am implementing OpenAI gym's cartpole problem using Deep Q-Learning (DQN). I followed tutorials (video and otherwise) and learned all about it. I implemented a code for myself and I thought it ...

reinforcement-learning q-learning dqn openai-gym

asked Aug 12 '20 at 0:42

Kashan

1011 bronze badge

0

votes

1answer

65 views

Deep Q-learning, how to set q-value of non-selected actions?

I am learning Deep Q-learning by applying it to a real world problem. I have been through some tutorials and papers available online but I counldn't figure out the solution for the following problem ...

neural-network deep-learning q-learning

asked Jul 21 '20 at 10:04

Rasoul

1012 bronze badges

1

vote

0answers

13 views

Wich activation function for DQL

After many research, I still can't find a neat answer about this question: When I found the loss of my state-action pair. I'm only backpropagating that loss true the network and setting all other ...

neural-network reinforcement-learning backpropagation activation-function q-learning

asked Jun 17 '20 at 12:26

Alexandre Martens

111 bronze badge

0

votes

1answer

106 views

Is this a valid stability concern/improvement for DQN/DDQN reinforcement training?

As you all know, DQN or DDQN are known for "unstable training". Let's use the well known "CartPole". The agent has to balance the stick and gets a reward of +1 per frame. You can reach the 195 ...

reinforcement-learning q-learning dqn

asked Jun 4 '20 at 22:31

laz

133 bronze badges

0

votes

1answer

56 views

How to save and load a Q-Learning Agent

I know this may sound nooby, but how do I save a Deep Q-Learning agent's progress? I mean when I close at i.e. episode 500 when my agent is trained and I restart (in my case a pygame) my agent is ...

python training q-learning

asked May 21 '20 at 12:06

Marc Vana

31 bronze badge

1

vote

0answers

81 views

Reward(t) vs. Reward(t+1) ? Reinforcement Learning, Q-learning

In "Reinforcement Learning, An Introduction; Richard S. Sutton and Andrew G. Barto" is written on page 70: 3.1 The Agent–Environment Interface At each time step t, the agent receives some ...

reinforcement-learning q-learning

asked Mar 19 '20 at 1:59

anna12345

212 bronze badges

1

vote

0answers

93 views

Q-learning, state transition, immediate rewards (trading logic)

I've been thinking about how to correctly calculate rewards for several weeks now. Here is a grid example: ...

python reinforcement-learning q-learning dqn

asked Mar 16 '20 at 5:53

anna12345

212 bronze badges

2

votes

1answer

32 views

Markov Decision Process representation

I'm attempting to model a simple process using a Markov Decision Process. Let $A$ be a set of $3$ actions : $ A \in \{b,s\}$. $T(s,a,s')$ represents the probability of if in state $s$ , take action $...

machine-learning reinforcement-learning q-learning markov-process

asked Feb 9 '20 at 15:07

blue-sky

1971 silver badge7 bronze badges

1

vote

0answers

16 views

help understanding deep Q learning algorithm from deep mind paper

I'm studying this paper https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf, and I have a question about proposed algorithm(it appears on page 7 of the paper): The problem I see is ...

q-learning

asked Feb 3 '20 at 15:03

burer

111 bronze badge

1

vote

1answer

28 views

How is the target_f updated in the Keras solution to the Deep Q-learning Cartpole/Gym algorithm?

There's a popular solution to the CartPole game using Keras and Deep Q-Learning: https://keon.github.io/deep-q-learning/ But there's a line of code that's confusing, this same question has been asked ...

deep-learning keras reinforcement-learning q-learning openai-gym

asked Feb 1 '20 at 13:17

David Goudet

132 bronze badges

2

votes

2answers

118 views

Deep Q Network gives same Q values and doesn't improve

I'm trying to build a deep Q network to play snake. I've run into an issue where the agent doesn't learn and its performance at the end of the training cycle is to repeatedly kill itself. After a bit ...

python deep-learning keras reinforcement-learning q-learning

asked Dec 30 '19 at 22:10

achandra03

594 bronze badges

2

votes

1answer

243 views

Would Deep Q Learning work for a finite horizon problem?

I want to apply Deep Q Learning to a problem, which has a clear finite horizon definition, like: $$V(s) = \mathbb{E}[r_1 + r_2]$$ Since the horizon is finite, I do not use reward discounting. My ...

deep-learning reinforcement-learning q-learning dqn

asked Dec 26 '19 at 21:32

Ufuk Can Bicici

2031 silver badge5 bronze badges

2

votes

0answers

49 views

Incentivizing curiosity in a sparse reward environment

I'm quite new to reinforcement learning, but have been exploring different kinds of architectures (DQN, dueling DQN, actor critic, etc.) and evaluating their ability to solve certain problems. The ...

machine-learning neural-network deep-learning reinforcement-learning q-learning

asked Dec 14 '19 at 19:07

jhfa

611 bronze badge

1

vote

0answers

20 views

Deep Q Learning - training slows down significantly

I'm trying to build a deep Q network to play snake. I designed the game so that the window is 600 by 600 and the snake's head moves 30 pixels each tick. I implemented the DQN algorithm with memory ...

machine-learning python reinforcement-learning q-learning

asked Nov 29 '19 at 2:51

achandra03

594 bronze badges

1

vote

0answers

177 views

Reducing the training time of an RL agent

I am trying to develop an rl agent using DQN algorithm.During training, the agent interacts with environment which is a simulated one.Each episode takes around 10 mins to run. This way if want my ...

reinforcement-learning q-learning dqn policy-gradients dynamic-programming

asked Sep 19 '19 at 8:34

cvg

4555 silver badges16 bronze badges

1

vote

1answer

100 views

How to formulate reward of an rl agent with two objectives

I have started learning reinforcement learning and trying to apply it for my use case. I am developing an rl agent which can maintain temperature at a particular value, and minimize the energy ...

reinforcement-learning q-learning dqn monte-carlo discounted-reward

asked Sep 17 '19 at 8:36

cvg

4555 silver badges16 bronze badges

1

vote

0answers

42 views

Keras high loss and high accuracy in gk bot with reinforcement learning?

I'm making goal-keeper bot in haxball game. It worked well when i trained less but i worked worse when i trained more. Last reinforcement state: 5160 episode - 4171281 steps - 0.05 epsilon: Last fit ...

python deep-learning keras reinforcement-learning q-learning

asked Sep 5 '19 at 17:40

SolutionLover

111 bronze badge

1

vote

1answer

330 views

Q-learning when minimising a total cost instead of maximising a total reward

I have a decision problem where the results are measured as a cost that I want to minimise. It seems like a good fit to Q-learning, but I am not sure how to adjust it to deal with a cost instead of a ...

reinforcement-learning q-learning markov-process

asked Jul 30 '19 at 12:14

EArwa

456 bronze badges

0

votes

1answer

64 views

Deep Q-Learning for physical quantity: q-values distribution not as expected

Setting I am trying to learn a specific physical quantity (radiance) inside a 3D scene with Deep Q-Learning. Just to give a quick overview, my agent shoots rays inside the scene: the reward is the ...

deep-learning reinforcement-learning q-learning dqn

asked Jul 29 '19 at 9:24

maurock

2542 silver badges10 bronze badges

1

vote

1answer

36 views

Why can't Policy Gradient Algorithm be seen as an Actor-Critic Method?

During the equation deducing in policy gradient algorithm(e.g., REINFORCE), we are actually using an expectancy of total reward, which we try to maximize. $$\overline{R_\theta}=E_{\tau\sim\pi_\theta}[...

machine-learning reinforcement-learning q-learning policy-gradients actor-critic

asked Jul 22 '19 at 9:54

KningTG

111 bronze badge

1

vote

1answer

776 views

DQN - target values vs action values?

I'm trying to understand the difference between target-values and action-values in Deep Q Networks. From what I understand, action-value tries to approximate the reward of a given action (at some ...

deep-learning reinforcement-learning q-learning

asked Jul 21 '19 at 17:42

Alter

1234 bronze badges

6

votes

1answer

1k views

Why could my DDQN get significantly worse after beating the game repeatedly?

I've been trying to train a DDQN to play OpenAI Gym's CartPole-v1, but found that although it starts off well and starts getting full score (500) repeatedly (at around 600 episodes in the pic below), ...

python deep-learning reinforcement-learning q-learning openai-gym

asked Jul 20 '19 at 8:06

Danny Tuppeny

1636 bronze badges

1

vote

1answer

416 views

Q table creation and update for dynamic action space

I am trying to implement a Q-learning algorithm for energy optimization. It is a finite MDP with states represented as 6 dimensional vectors of integers. The number of discrete values in each index of ...

q-learning

asked Jul 16 '19 at 14:41

EArwa

456 bronze badges

1

vote

2answers

635 views

Alternative approach for Q-Learning

I have a question related to an alterative Q-Learning approach. I'd like to know if this already exists and I am not aware of it, or it doesn't exist because there are theoretical problems behind it. ...

machine-learning reinforcement-learning q-learning

asked Jun 22 '19 at 8:23

maurock

2542 silver badges10 bronze badges

Questions tagged [q-learning]

Related Tags