Newest 'q-learning' Questions

2

votes

1answer

39 views

Does convergence equal learning in Deep Q-learning?

In my current research project I'm using the Deep Q-learning algorithm. The setup is as follows: I'm training the model (using Deep Q-learning) on a static dataset made up of experiences extracted ...

asked Feb 21 at 16:00

Aeryan

435 bronze badges

0

votes

0answers

10 views

How can my loss be stable while the gradient keeps growing?

I have been working on an Offline/Batch Reinforcement Learning problem where I am using a BCQ-DDQN model as a Q-table. The model input is a state of 8 dimensions, and the output is a vector of Q-...

deep-learning reinforcement-learning loss-function q-learning gradient

asked Jan 27 at 13:23

MetaHG

212 bronze badges

0

votes

1answer

36 views

How to report results of RL with high variance?

I run Q-learning and SARSA algortihms on the same problem but the results fluctuate heavily and when I draw them, there is no smooth graph. How should I repost the results? I run algorithms for 500 ...

reinforcement-learning q-learning

asked Jan 6 at 19:41

Katatonia

1911 silver badge4 bronze badges

1

vote

0answers

12 views

Tensorflow deep Q neural network wont learn with more than 1 hidden layer

I've been using TensorFlow in Python with a very simple deep Q neural network (NN) to play a very simple game. The game is in an array [0, 0, 0, 0] and by picking one of the positions 0, 1, 2, 3, the ...

machine-learning deep-learning q-learning

asked Dec 30 '20 at 18:32

Oliver Nicholls

111 bronze badge

2

votes

0answers

35 views

Offline/Batch Reinforcement Learning: Doubly Robust Off-policy Estimator takes huge values

Context: My team and I are working on a RL problem for a specific application. We have data collected from user interactions (states, actions, etc.). It is too costly for us to emulate agents. We ...

machine-learning dataset reinforcement-learning q-learning estimators

asked Dec 4 '20 at 14:41

MetaHG

212 bronze badges

0

votes

0answers

29 views

Cannot solve FrozenLake with QLearning and slippery mode enabled

I'm trying to solve the open-ai gym frozen lake problem with a basic Q learning table and it works quite well with is_slippery=False. But it cannot find a solution ...

python q-learning openai-gym

asked Nov 15 '20 at 18:01

Marc

1214 bronze badges

1

vote

1answer

46 views

Definition of the Q* function in reinforcement learning

I'm making my way through Sutton's Introduction to Reinforcement Learning. He gives the definition of the $q_*$ function as follows $$ q_*(a) = \mathbf{E}[R_t | A_t = a] $$ where $A_t$ is the action ...

machine-learning reinforcement-learning q-learning

asked Oct 10 '20 at 22:37

marlineer43

132 bronze badges

2

votes

0answers

20 views

Reference implementation of q-learning in Python

I'm a machine learning newbie, trying to learn Q-learning. I read a few texts and I get the general gist, but what I'd really love to see is a simple example of a Q-learning algorithm in Python that I ...

machine-learning python q-learning implementation

asked Sep 5 '20 at 16:15

Ram Rachum

1613 bronze badges

0

votes

1answer

29 views

Different Initial Q-Values in Q-Learning

When working with Q-Learning, what is the difference between having a Q_0(a) with all values zero, random or optimistic?

reinforcement-learning q-learning

asked Aug 30 '20 at 10:25

Giulia

1

0

votes

1answer

21 views

When should I use normal Q learning over a DQN?

From this article here, it says that using a tabular Q function is less scalable than a deep Q network. I assume that this means that the Q table approach works for some environments, but once they ...

reinforcement-learning q-learning

asked Aug 27 '20 at 0:09

jasooney23

31 bronze badge

0

votes

0answers

26 views

My Deep Q-Learning Network does not learn for OpenAI gym's cartpole problem

I am implementing OpenAI gym's cartpole problem using Deep Q-Learning (DQN). I followed tutorials (video and otherwise) and learned all about it. I implemented a code for myself and I thought it ...

reinforcement-learning q-learning dqn openai-gym

asked Aug 12 '20 at 0:42

Kashan

1011 bronze badge

0

votes

1answer

52 views

Deep Q-learning, how to set q-value of non-selected actions?

I am learning Deep Q-learning by applying it to a real world problem. I have been through some tutorials and papers available online but I counldn't figure out the solution for the following problem ...

neural-network deep-learning q-learning

asked Jul 21 '20 at 10:04

Rasoul

1012 bronze badges

1

vote

0answers

13 views

Wich activation function for DQL

After many research, I still can't find a neat answer about this question: When I found the loss of my state-action pair. I'm only backpropagating that loss true the network and setting all other ...

neural-network reinforcement-learning backpropagation activation-function q-learning

asked Jun 17 '20 at 12:26

Alexandre Martens

111 bronze badge

0

votes

1answer

70 views

Is this a valid stability concern/improvement for DQN/DDQN reinforcement training?

As you all know, DQN or DDQN are known for "unstable training". Let's use the well known "CartPole". The agent has to balance the stick and gets a reward of +1 per frame. You can reach the 195 ...

reinforcement-learning q-learning dqn

asked Jun 4 '20 at 22:31

laz

133 bronze badges

0

votes

1answer

40 views

How to save and load a Q-Learning Agent

I know this may sound nooby, but how do I save a Deep Q-Learning agent's progress? I mean when I close at i.e. episode 500 when my agent is trained and I restart (in my case a pygame) my agent is ...

python training q-learning

asked May 21 '20 at 12:06

Marc Vana

31 bronze badge

0

votes

0answers

26 views

First Simple DQN not learning to navigate maze

So I am currently attempting to write my first DQN implementation, where the aim is for the agent to learn to navigate the board from the top left to the bottom right while avoiding the hole right in ...

neural-network deep-learning reinforcement-learning q-learning

asked May 8 '20 at 17:34

Krothagon

112 bronze badges

0

votes

0answers

21 views

Can we use Q-Learning (or RL in general) for this problem?

Let's say that we have an algorithm that given a dataset point, it runs some analysis on it and returns the results. The algorithm has a user-defined parameter X that affects the run-time of the ...

machine-learning reinforcement-learning q-learning

asked May 5 '20 at 15:35

Omid

1

0

votes

0answers

11 views

Reward Function for a model-free MDP

I am trying to build a program which has to decide in a completely stochastic environment. So it has to be model-free and Q-learning is suitable for that. I just have one problem, my rewards are not ...

reinforcement-learning q-learning

asked Apr 23 '20 at 6:49

Ammar

1

0

votes

0answers

8 views

For off policy reinforcement learning how different can the current policy be from the policies which generated the data

Say we have two policies and we use one to generate data with. We now want to use this data to optimize the second policy (the two policies are defined with the same input and output space but with ...

deep-learning reinforcement-learning q-learning policy-gradients

asked Apr 22 '20 at 21:50

Azerila

1011 bronze badge

0

votes

0answers

6 views

Reward engineering to replace single terminal reward (exponential utility of terminal wealth)

My goal is to use reinforcement learning to train the agent (the trader) to maximize the exponential utility of his P&L (profit and loss) at a terminal time T. Therefore the natural formulation of ...

reinforcement-learning optimization q-learning discounted-reward

asked Apr 21 '20 at 11:06

BGa

11 bronze badge

0

votes

0answers

21 views

How are n dimensional vectors state vectors represented in Q Learning?

Using this code: ...

reinforcement-learning beginner q-learning

asked Apr 14 '20 at 19:46

blue-sky

2071 silver badge7 bronze badges

1

vote

0answers

74 views

Reward(t) vs. Reward(t+1) ? Reinforcement Learning, Q-learning

In "Reinforcement Learning, An Introduction; Richard S. Sutton and Andrew G. Barto" is written on page 70: 3.1 The Agent–Environment Interface At each time step t, the agent receives some ...

reinforcement-learning q-learning

asked Mar 19 '20 at 1:59

anna12345

212 bronze badges

0

votes

0answers

194 views

Index tensor must have same dimensions as input tensor

I am trying to train a DQN to do optimal energy scheduling. Each state comes as a vector of 4 variables (represented by floats) saved in the replay memory as a state tensor, each action is an integer ...

deep-learning reinforcement-learning q-learning dqn

asked Mar 17 '20 at 20:29

EArwa

356 bronze badges

1

vote

0answers

92 views

Q-learning, state transition, immediate rewards (trading logic)

I've been thinking about how to correctly calculate rewards for several weeks now. Here is a grid example: ...

python reinforcement-learning q-learning dqn

asked Mar 16 '20 at 5:53

anna12345

212 bronze badges

0

votes

0answers

16 views

find the parameter of model with Q learning

I have a question with regard to Q learning. I am a beginner in Q learning. Every example that I saw is related to the environment that the goal is assigned to a place. (like cliff walk that we know ...

reinforcement-learning q-learning learning

asked Mar 2 '20 at 17:44

Hasan

11 bronze badge

2

votes

1answer

31 views

Markov Decision Process representation

I'm attempting to model a simple process using a Markov Decision Process. Let $A$ be a set of $3$ actions : $ A \in \{b,s\}$. $T(s,a,s')$ represents the probability of if in state $s$ , take action $...

machine-learning reinforcement-learning q-learning markov-process

asked Feb 9 '20 at 15:07

blue-sky

2071 silver badge7 bronze badges

1

vote

0answers

16 views

help understanding deep Q learning algorithm from deep mind paper

I'm studying this paper https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf, and I have a question about proposed algorithm(it appears on page 7 of the paper): The problem I see is ...

q-learning

asked Feb 3 '20 at 15:03

burer

111 bronze badge

0

votes

1answer

23 views

How is the target_f updated in the Keras solution to the Deep Q-learning Cartpole/Gym algorithm?

There's a popular solution to the CartPole game using Keras and Deep Q-Learning: https://keon.github.io/deep-q-learning/ But there's a line of code that's confusing, this same question has been asked ...

deep-learning keras reinforcement-learning q-learning openai-gym

asked Feb 1 '20 at 13:17

David Goudet

31 bronze badge

2

votes

2answers

115 views

Deep Q Network gives same Q values and doesn't improve

I'm trying to build a deep Q network to play snake. I've run into an issue where the agent doesn't learn and its performance at the end of the training cycle is to repeatedly kill itself. After a bit ...

python deep-learning keras reinforcement-learning q-learning

asked Dec 30 '19 at 22:10

achandra03

494 bronze badges

2

votes

1answer

200 views

Would Deep Q Learning work for a finite horizon problem?

I want to apply Deep Q Learning to a problem, which has a clear finite horizon definition, like: $$V(s) = \mathbb{E}[r_1 + r_2]$$ Since the horizon is finite, I do not use reward discounting. My ...

deep-learning reinforcement-learning q-learning dqn

asked Dec 26 '19 at 21:32

Ufuk Can Bicici

2031 silver badge5 bronze badges

2

votes

0answers

44 views

Incentivizing curiosity in a sparse reward environment

I'm quite new to reinforcement learning, but have been exploring different kinds of architectures (DQN, dueling DQN, actor critic, etc.) and evaluating their ability to solve certain problems. The ...

machine-learning neural-network deep-learning reinforcement-learning q-learning

asked Dec 14 '19 at 19:07

jhfa

611 bronze badge

1

vote

0answers

20 views

Deep Q Learning - training slows down significantly

I'm trying to build a deep Q network to play snake. I designed the game so that the window is 600 by 600 and the snake's head moves 30 pixels each tick. I implemented the DQN algorithm with memory ...

machine-learning python reinforcement-learning q-learning

asked Nov 29 '19 at 2:51

achandra03

494 bronze badges

1

vote

0answers

147 views

Reducing the training time of an RL agent

I am trying to develop an rl agent using DQN algorithm.During training, the agent interacts with environment which is a simulated one.Each episode takes around 10 mins to run. This way if want my ...

reinforcement-learning q-learning dqn policy-gradients dynamic-programming

asked Sep 19 '19 at 8:34

cvg

4555 silver badges16 bronze badges

1

vote

1answer

85 views

How to formulate reward of an rl agent with two objectives

I have started learning reinforcement learning and trying to apply it for my use case. I am developing an rl agent which can maintain temperature at a particular value, and minimize the energy ...

reinforcement-learning q-learning dqn monte-carlo discounted-reward

asked Sep 17 '19 at 8:36

cvg

4555 silver badges16 bronze badges

1

vote

0answers

40 views

Keras high loss and high accuracy in gk bot with reinforcement learning?

I'm making goal-keeper bot in haxball game. It worked well when i trained less but i worked worse when i trained more. Last reinforcement state: 5160 episode - 4171281 steps - 0.05 epsilon: Last fit ...

python deep-learning keras reinforcement-learning q-learning

asked Sep 5 '19 at 17:40

SolutionLover

111 bronze badge

1

vote

1answer

251 views

Q-learning when minimising a total cost instead of maximising a total reward

I have a decision problem where the results are measured as a cost that I want to minimise. It seems like a good fit to Q-learning, but I am not sure how to adjust it to deal with a cost instead of a ...

reinforcement-learning q-learning markov-process

asked Jul 30 '19 at 12:14

EArwa

356 bronze badges

0

votes

1answer

60 views

Deep Q-Learning for physical quantity: q-values distribution not as expected

Setting I am trying to learn a specific physical quantity (radiance) inside a 3D scene with Deep Q-Learning. Just to give a quick overview, my agent shoots rays inside the scene: the reward is the ...

deep-learning reinforcement-learning q-learning dqn

asked Jul 29 '19 at 9:24

maurock

2442 silver badges10 bronze badges

1

vote

1answer

35 views

Why can't Policy Gradient Algorithm be seen as an Actor-Critic Method?

During the equation deducing in policy gradient algorithm(e.g., REINFORCE), we are actually using an expectancy of total reward, which we try to maximize. $$\overline{R_\theta}=E_{\tau\sim\pi_\theta}[...

machine-learning reinforcement-learning q-learning policy-gradients actor-critic

asked Jul 22 '19 at 9:54

KningTG

111 bronze badge

1

vote

1answer

657 views

DQN - target values vs action values?

I'm trying to understand the difference between target-values and action-values in Deep Q Networks. From what I understand, action-value tries to approximate the reward of a given action (at some ...

deep-learning reinforcement-learning q-learning

asked Jul 21 '19 at 17:42

Alter

1234 bronze badges

7

votes

1answer

1k views

Why could my DDQN get significantly worse after beating the game repeatedly?

I've been trying to train a DDQN to play OpenAI Gym's CartPole-v1, but found that although it starts off well and starts getting full score (500) repeatedly (at around 600 episodes in the pic below), ...

python deep-learning reinforcement-learning q-learning openai-gym

asked Jul 20 '19 at 8:06

Danny Tuppeny

1736 bronze badges

1

vote

1answer

350 views

Q table creation and update for dynamic action space

I am trying to implement a Q-learning algorithm for energy optimization. It is a finite MDP with states represented as 6 dimensional vectors of integers. The number of discrete values in each index of ...

q-learning

asked Jul 16 '19 at 14:41

EArwa

356 bronze badges

1

vote

2answers

541 views

Alternative approach for Q-Learning

I have a question related to an alterative Q-Learning approach. I'd like to know if this already exists and I am not aware of it, or it doesn't exist because there are theoretical problems behind it. ...

machine-learning reinforcement-learning q-learning

asked Jun 22 '19 at 8:23

maurock

2442 silver badges10 bronze badges

2

votes

1answer

750 views

Difference between Dueling DQN and Double DQN?

I have read some articles, but still can not figure out the difference between the Dueling DQN and Double DQN? What exactly is the difference between them? Also, Does Dueling DQN need to be built on ...

reinforcement-learning q-learning dqn deep-network

asked May 31 '19 at 17:46

Edamame

2,0555 gold badges12 silver badges25 bronze badges

2

votes

0answers

130 views

Reinforcement Learning using PPO2 in openai gym retro, mario not learning the clear the easy episode

I am training mario game in retro using ppo2 baselines for some time. I have tried level3 and level1 too. But even after full training when I play using saved checkpoints, the mario is not able to ...

reinforcement-learning q-learning openai-gym

asked May 11 '19 at 10:46

Sandeep Bhutani

6533 silver badges15 bronze badges

1

vote

0answers

78 views

Intuition behind the loss function in Deep Q learning?

I'm currently following a tutorial but I got stuck at the deep Q learning model. According to my understanding of neural networks they predict an approximate function for the inputs given with the ...

deep-learning reinforcement-learning loss-function q-learning

asked May 5 '19 at 12:34

anjani

112 bronze badges

2

votes

1answer

25 views

If the set of all possible states changes each time, how can Q-learning “learn” anything?

I found this resource that explains q-learning with a very simple example. Make it a 2D problem, a rectangle instead of a line, and it's still simple. The only difference is that now there are 2 more ...

reinforcement-learning q-learning

asked May 4 '19 at 21:40

Pigna

1633 bronze badges

2

votes

1answer

233 views

Representing state in Q-Learning

I have a fairly simple game in which I wish to use Q-learning to train an agent, but I have some questions regarding state representation. I'm new to RL so bare with me: If you have a game where you ...

reinforcement-learning beginner q-learning

asked May 4 '19 at 9:09

Ross Kohler

211 bronze badge

0

votes

1answer

976 views

Q-Learning experience replay: how to feed the neural network?

I'm trying to replicate the DQN Atari experiment. Actually my DQN isn't performing well; checking another one's codes, I saw something about experience replay which I don't understand. First, when you ...

python reinforcement-learning q-learning dqn keras-rl

asked Apr 11 '19 at 14:13

Joaquin

12 bronze badges

3

votes

1answer

661 views

DQN fails to find optimal policy

Based on DeepMind publication, I've recreated the environment and I am trying to make the DQN find and converge to an optimal policy. The task of an agent is to learn how to sustainably collect apples ...

reinforcement-learning q-learning dqn convergence deepmind

asked Apr 1 '19 at 1:23

macwiatrak

313 bronze badges

1

vote

0answers

24 views

Is reward accumulated during a play iteration when performing SARSA?

I've been having issue with getting my DQN to converge to a good solution for snake. Regardless of the different types of reward functions I've tried, it seems that the snake is indefinitely going ...

machine-learning deep-learning reinforcement-learning q-learning dqn

asked Mar 29 '19 at 1:44

DevarakondaV

314 bronze badges

Questions tagged [q-learning]

Related Tags