Newest 'q-learning' Questions

2

votes

2answers

99 views

Deep Q Network gives same Q values and doesn't improve

I'm trying to build a deep Q network to play snake. I've run into an issue where the agent doesn't learn and its performance at the end of the training cycle is to repeatedly kill itself. After a bit ...

asked Dec 30 '19 at 22:10

achandra03

414 bronze badges

2

votes

1answer

52 views

Would Deep Q Learning work for a finite horizon problem?

I want to apply Deep Q Learning to a problem, which has a clear finite horizon definition, like: $$V(s) = \mathbb{E}[r_1 + r_2]$$ Since the horizon is finite, I do not use reward discounting. My ...

deep-learning reinforcement-learning q-learning dqn

asked Dec 26 '19 at 21:32

Ufuk Can Bicici

1935 bronze badges

2

votes

0answers

24 views

Incentivizing curiosity in a sparse reward environment

I'm quite new to reinforcement learning, but have been exploring different kinds of architectures (DQN, dueling DQN, actor critic, etc.) and evaluating their ability to solve certain problems. The ...

machine-learning neural-network deep-learning reinforcement-learning q-learning

asked Dec 14 '19 at 19:07

jhfa

611 bronze badge

1

vote

0answers

13 views

Deep Q Learning - training slows down significantly

I'm trying to build a deep Q network to play snake. I designed the game so that the window is 600 by 600 and the snake's head moves 30 pixels each tick. I implemented the DQN algorithm with memory ...

machine-learning python reinforcement-learning q-learning

asked Nov 29 '19 at 2:51

achandra03

414 bronze badges

0

votes

0answers

17 views

What reinforcement learning algorithm to choose for self-driving car

I have a car that has three sensors at the front. Using these sensors only I want to let it learn to drive on a track. I'm new to reinforcement learning, but I was thinking about using the Q-learning ...

machine-learning reinforcement-learning q-learning

asked Nov 22 '19 at 8:58

Zarif

101

0

votes

0answers

25 views

Applying Reinforcement Learning in the following scenario

I'm working on a scenario/environment where I have a simulation that provides an arrangement or results of the simulation that has data in a format of samples in vectors(x,y,z,N). Let's say it maps ...

reinforcement-learning q-learning markov-process

asked Nov 18 '19 at 14:06

RandomGuest

1

0

votes

0answers

31 views

Reducing the training time of an RL agent

I am trying to develop an rl agent using DQN algorithm.During training, the agent interacts with environment which is a simulated one.Each episode takes around 10 mins to run. This way if want my ...

reinforcement-learning q-learning dqn policy-gradients dynamic-programming

asked Sep 19 '19 at 8:34

cvg

36711 bronze badges

0

votes

0answers

50 views

Prioritized Experience Replay - for whole episodes

I want to use Prioritized Experience Replay for whole episodes, instead for single transitions. What's the best way to define the priorities - as episodes can be of different lengths? Personally I ...

machine-learning reinforcement-learning q-learning

asked Sep 19 '19 at 0:37

ScientiaEtVeritas

1416 bronze badges

1

vote

1answer

29 views

How to formulate reward of an rl agent with two objectives

I have started learning reinforcement learning and trying to apply it for my use case. I am developing an rl agent which can maintain temperature at a particular value, and minimize the energy ...

reinforcement-learning q-learning dqn monte-carlo discounted-reward

asked Sep 17 '19 at 8:36

cvg

36711 bronze badges

1

vote

0answers

26 views

Keras high loss and high accuracy in gk bot with reinforcement learning?

I'm making goal-keeper bot in haxball game. It worked well when i trained less but i worked worse when i trained more. Last reinforcement state: 5160 episode - 4171281 steps - 0.05 epsilon: Last fit ...

python deep-learning keras reinforcement-learning q-learning

asked Sep 5 '19 at 17:40

beta

111 bronze badge

1

vote

1answer

45 views

Q-learning when minimising a total cost instead of maximising a total reward

I have a decision problem where the results are measured as a cost that I want to minimise. It seems like a good fit to Q-learning, but I am not sure how to adjust it to deal with a cost instead of a ...

reinforcement-learning q-learning markov-process

asked Jul 30 '19 at 12:14

EArwa

255 bronze badges

0

votes

1answer

47 views

Deep Q-Learning for physical quantity: q-values distribution not as expected

Setting I am trying to learn a specific physical quantity (radiance) inside a 3D scene with Deep Q-Learning. Just to give a quick overview, my agent shoots rays inside the scene: the reward is the ...

deep-learning reinforcement-learning q-learning dqn

asked Jul 29 '19 at 9:24

maurocomi

1941 silver badge7 bronze badges

0

votes

0answers

15 views

What's the relationship amoung Temporal Difference and Policy Gradient, Deep Q learning?

I've gone through some comparisons between MC and TD to estimate $V_{\pi_\theta}(s)$. However, it seems to be not only a method to estimate V, but also a mechanism that can improve the update ...

machine-learning reinforcement-learning q-learning policy-gradients

asked Jul 22 '19 at 10:46

KningTG

111 bronze badge

1

vote

1answer

17 views

Why can't Policy Gradient Algorithm be seen as an Actor-Critic Method?

During the equation deducing in policy gradient algorithm(e.g., REINFORCE), we are actually using an expectancy of total reward, which we try to maximize. $$\overline{R_\theta}=E_{\tau\sim\pi_\theta}[...

machine-learning reinforcement-learning q-learning policy-gradients actor-critic

asked Jul 22 '19 at 9:54

KningTG

111 bronze badge

1

vote

1answer

142 views

DQN - target values vs action values?

I'm trying to understand the difference between target-values and action-values in Deep Q Networks. From what I understand, action-value tries to approximate the reward of a given action (at some ...

deep-learning reinforcement-learning q-learning

asked Jul 21 '19 at 17:42

Alter

1234 bronze badges

3

votes

1answer

174 views

Why could my DDQN get significantly worse after beating the game repeatedly?

I've been trying to train a DDQN to play OpenAI Gym's CartPole-v1, but found that although it starts off well and starts getting full score (500) repeatedly (at around 600 episodes in the pic below), ...

python deep-learning reinforcement-learning q-learning openai-gym

asked Jul 20 '19 at 8:06

Danny Tuppeny

1335 bronze badges

1

vote

1answer

142 views

Q table creation and update for dynamic action space

I am trying to implement a Q-learning algorithm for energy optimization. It is a finite MDP with states represented as 6 dimensional vectors of integers. The number of discrete values in each index of ...

q-learning

asked Jul 16 '19 at 14:41

EArwa

255 bronze badges

1

vote

2answers

160 views

Alternative approach for Q-Learning

I have a question related to an alterative Q-Learning approach. I'd like to know if this already exists and I am not aware of it, or it doesn't exist because there are theoretical problems behind it. ...

machine-learning reinforcement-learning q-learning

asked Jun 22 '19 at 8:23

maurocomi

1941 silver badge7 bronze badges

1

vote

1answer

300 views

Difference between Dueling DQN and Double DQN?

I have read some articles, but still can not figure out the difference between the Dueling DQN and Double DQN? What exactly is the difference between them? Also, Does Dueling DQN need to be built on ...

reinforcement-learning q-learning dqn deep-network

asked May 31 '19 at 17:46

Edamame

1,4453 gold badges11 silver badges20 bronze badges

0

votes

0answers

5 views

Deep Q Learning state dimensionality

How important is the dimensionality of each state for Deep Q Learning? I have a set of 15 unique playing cards from a deck of 52 playing cards. A given state is represented by the respective card ...

deep-learning dimensionality-reduction q-learning

asked May 16 '19 at 6:58

Harry Stuart

1235 bronze badges

2

votes

0answers

81 views

Reinforcement Learning using PPO2 in openai gym retro, mario not learning the clear the easy episode

I am training mario game in retro using ppo2 baselines for some time. I have tried level3 and level1 too. But even after full training when I play using saved checkpoints, the mario is not able to ...

reinforcement-learning q-learning openai-gym

asked May 11 '19 at 10:46

Sandeep Bhutani

3721 silver badge12 bronze badges

0

votes

0answers

17 views

Reinforcement Learning - Q Learning - Number of Steps to Decrease?

I have an implementation of the Q-Learning algorithm intended to solve the racetrack problem. I have noticed that the initial amount of steps needed to solve the problem is somewhere between 3000-...

machine-learning reinforcement-learning algorithms q-learning

asked May 9 '19 at 1:18

wundermahn

1135 bronze badges

1

vote

0answers

55 views

Intuition behind the loss function in Deep Q learning?

I'm currently following a tutorial but I got stuck at the deep Q learning model. According to my understanding of neural networks they predict an approximate function for the inputs given with the ...

deep-learning reinforcement-learning loss-function q-learning

asked May 5 '19 at 12:34

anjani

112 bronze badges

2

votes

1answer

25 views

If the set of all possible states changes each time, how can Q-learning “learn” anything?

I found this resource that explains q-learning with a very simple example. Make it a 2D problem, a rectangle instead of a line, and it's still simple. The only difference is that now there are 2 more ...

reinforcement-learning q-learning

asked May 4 '19 at 21:40

Pigna

1633 bronze badges

2

votes

1answer

35 views

Representing state in Q-Learning

I have a fairly simple game in which I wish to use Q-learning to train an agent, but I have some questions regarding state representation. I'm new to RL so bare with me: If you have a game where you ...

reinforcement-learning beginner q-learning

asked May 4 '19 at 9:09

Ross Kohler

211 bronze badge

0

votes

0answers

58 views

Why is the reward fluctuating for Double Q-Learning?

I am trying to implement Double Q-Learning using neural networks from the Keras library. When I first tried Simple DQN, the graph of the reward was fluctuating a lot so, I implemented a Double DQN. ...

deep-learning reinforcement-learning q-learning

asked Apr 20 '19 at 8:19

Gauranga Das

12 bronze badges

0

votes

1answer

342 views

Q-Learning experience replay: how to feed the neural network?

I'm trying to replicate the DQN Atari experiment. Actually my DQN isn't performing well; checking another one's codes, I saw something about experience replay which I don't understand. First, when you ...

python reinforcement-learning q-learning dqn keras-rl

asked Apr 11 '19 at 14:13

Joaquin

12 bronze badges

0

votes

0answers

22 views

Different algorithms categorized in reinforcement learning

(Originally asked at cross validated forum: https://stats.stackexchange.com/questions/401615/different-algorithms-categorized-in-reinforcement-learning) For some time I am going through reinforcement ...

reinforcement-learning q-learning

asked Apr 8 '19 at 20:28

Sandeep Bhutani

3721 silver badge12 bronze badges

2

votes

1answer

372 views

DQN fails to find optimal policy

Based on DeepMind publication, I've recreated the environment and I am trying to make the DQN find and converge to an optimal policy. The task of an agent is to learn how to sustainably collect apples ...

reinforcement-learning q-learning dqn convergence deepmind

asked Apr 1 '19 at 1:23

macwiatrak

213 bronze badges

0

votes

0answers

15 views

Is reward accumulated during a play iteration when performing SARSA?

I've been having issue with getting my DQN to converge to a good solution for snake. Regardless of the different types of reward functions I've tried, it seems that the snake is indefinitely going ...

machine-learning deep-learning reinforcement-learning q-learning dqn

asked Mar 29 '19 at 1:44

DevarakondaV

214 bronze badges

0

votes

1answer

73 views

Openai Spaces for a modified environment

I have a 2-dimensional array of normalized data. I am using space = np.array([0,1,...366],[0,0.000001,.....1]) I need to fit this as an observation space in ...

python deep-learning reinforcement-learning q-learning openai-gym

asked Mar 17 '19 at 16:29

Karthik Rajkumar

556 bronze badges

1

vote

0answers

11 views

Q learning transition matrix trouble

First, apologies if this isn't the right forum for my question -- let me know if there's a better place. I'm trying to implement Q learning on a two-dimensional grid world, where the agent has four ...

machine-learning q-learning

asked Jan 30 '19 at 1:14

Elizabeth Han

1113 bronze badges

0

votes

0answers

23 views

Q-Learning where some actions are more difficult to predict than others

I'm trying to train a deep Q network to optimize play in a game. For simplicity, let's say my game only has two actions, A and B. The reward distribution for A is somewhat uniform in the range -1 to ...

machine-learning deep-learning reinforcement-learning q-learning

asked Jan 29 '19 at 20:25

ahbutfore

211 bronze badge

0

votes

1answer

48 views

Why is “next state” kept in RL experience replay?

Following this explanation on what is experience replay (and others), I noticed an experience element is defined as $e_t = (s_t,a_t,r_t,s_{t+1})$ My question is, why do we need the ...

machine-learning reinforcement-learning q-learning policy-gradients

asked Jan 23 '19 at 18:03

Gulzar

1338 bronze badges

0

votes

0answers

11 views

Is every value-iteration off-policy DP is a Q-learning?

Is it true to say that every value-iteration off-policy DP is a Q-learning technique. or there are many more specific definition for it?

q-learning

asked Jan 22 '19 at 19:29

user10296606

8661 gold badge4 silver badges15 bronze badges

1

vote

0answers

24 views

What is the meaning of the Variant Q-learning and To what INPUT and OUTPUT refer? in Abstract of DeepMind DQN paper 2013

-INPUT and OUTPUT OF ATARI DQN: In the abstract paragraph of the DQN work by DeepMind https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf it has written: " We present the first deep learning model to ...

deep-learning reinforcement-learning q-learning dqn deep-network

asked Jan 22 '19 at 10:36

user10296606

8661 gold badge4 silver badges15 bronze badges

2

votes

1answer

779 views

What is the difference between dynamic programming and Q-learning?

What is the difference between the DP-based algorithm and Q-learning?

reinforcement-learning q-learning dynamic-programming

asked Jan 21 '19 at 5:52

user10296606

8661 gold badge4 silver badges15 bronze badges

0

votes

1answer

69 views

How to represent an image as state in a Q-table

I'm trying to do Q-learning with the Atari games using the gym python's package. I want to use the image as the state of my algorithm, but I came up with a doubt: ...

reinforcement-learning q-learning openai-gym

asked Dec 31 '18 at 18:22

Neoares

1435 bronze badges

0

votes

1answer

52 views

How does Q-Learning deal with mixed strategies?

I'm trying to understand how Q-learning deals with games where the optimal policy is a mixed strategy. The Bellman equation says that you should choose $max_a(Q(s,a))$ but this implies a single unique ...

machine-learning reinforcement-learning q-learning

asked Dec 20 '18 at 17:48

Thomas Johnson

5051 gold badge3 silver badges10 bronze badges

1

vote

1answer

72 views

Why not use max(returns) instead of average(returns) in off-policy Monte Carlo control?

As I understand it, in reinforcement learning, off-policy Monte Carlo control is when the state-action value function $Q(s,a)$ is estimated as a weighted average of the observed returns. However, in ...

reinforcement-learning q-learning monte-carlo

asked Dec 20 '18 at 10:52

Meldgaard

112 bronze badges

1

vote

1answer

272 views

Reinforcement learning: negative reward (punish) illegal actions?

If you train an agent using reinforcement learning (with Q-function in this case), should you give a negative reward (punish) if the agent proposes illegal actions for the presented state? I guess ...

machine-learning reinforcement-learning q-learning

asked Dec 2 '18 at 10:04

BigBadMe

5252 silver badges14 bronze badges

1

vote

1answer

1k views

IndexError: index 804 is out of bounds for axis 0 with size 800

i installed a self driving car project from superdatascience site , when i open the map using terminal after a while the map window close up or it closes directly after i maximize the map window and ...

reinforcement-learning python q-learning

asked Nov 5 '18 at 15:05

R.SH

1

vote

1answer

51 views

Will reinforcement learning work if states wont get repeated again?

I am working on a information retrieval model where the user enters a query and the model has to retrieve 3 most relevant FAQ pairs.I am collecting implicit feedback in terms of page clicks etc.What I ...

machine-learning deep-learning reinforcement-learning q-learning dqn

asked Oct 24 '18 at 11:57

Charan Reddy

1111 bronze badge

1

vote

0answers

92 views

Dueling DQN - Calculation of Q-value

I'm trying to implement a Double Dueling DQN on LunarLander and I'm facing an issue as my model is not learning so I'm trying to debug the graph and this leads me to a question regarding the ...

reinforcement-learning q-learning

asked Oct 19 '18 at 15:53

Nicolas M.

1115 bronze badges

1

vote

1answer

74 views

What's going wrong with my Tic Tac Toe Q-Learning Alghoritm?

my Q-learning alghoritm currently choices the "sub optimal" option and not the best one. ...

python q-learning

asked Oct 12 '18 at 20:11

CuriousPanda

113 bronze badges

1

vote

1answer

251 views

What is the immediate reward in value iteration?

Suppose you're given an MDP where rewards are attributed for reaching a state, independently of the action. Then when doing value iteration: $$ V_{i+1} = \max_a \sum_{s'} P_a(s,s') (R_a(s,s') + \...

reinforcement-learning q-learning

asked Oct 8 '18 at 14:38

Dominus

456 bronze badges

5

votes

1answer

3k views

Reinforcement learning: decreasing loss without increasing reward

I'm trying to solve OpenAI Gym's LunarLander-v2. I'm using the Deep Q-Learning algorithm. I have tried various hyperparameters, but I can't get a good score. Generally the loss decreases over many ...

reinforcement-learning q-learning

asked Sep 4 '18 at 12:06

Atuos

2172 silver badges5 bronze badges

2

votes

1answer

3k views

RL Advantage function why A = Q-V instead of A=V-Q?

In RL Course by David Silver - Lecture 7: Policy Gradient Methods, David explains what an Advantage function is, and how it's the difference between Q(s,a) and the V(s) Preliminary, from this post: ...

reinforcement-learning q-learning variance

asked Sep 1 '18 at 3:08

Kari

1,7147 silver badges30 bronze badges

2

votes

1answer

65 views

How we can have RF-QLearning or SVR-QLearning (Combine these algorithm with a Q-Learning )

How we can have RF-QLearning or SVR-QLearning (Combine these algorithm with a Q-Learning )? I want to replace the DNN section of Qlearning with a RF or SVR but the problem is that there is no clear ...

random-forest reinforcement-learning q-learning dqn svr

asked Aug 31 '18 at 18:01

user10296606

8661 gold badge4 silver badges15 bronze badges

1

vote

0answers

248 views

Deep Q-Learning with large number of actions

I'm using DQN with large number of actions in [0, 10000, step = 1000]. This means I have an action space of size 11 (including 0 and 10000). Action space is still discrete. My problem is that, instead ...

machine-learning deep-learning reinforcement-learning q-learning dqn

asked Aug 16 '18 at 6:04

Krishna Nevase

456 bronze badges

Questions tagged [q-learning]

Related Tags