Newest 'q-learning' Questions

1

vote

1answer

30 views

Alternative approach for Q-Learning

I have a question related to an alterative Q-Learning approach. I'd like to know if this already exists and I am not aware of it, or it doesn't exist because there are theoretical problems behind it. ...

asked Jun 22 at 8:23

maurocomi

547 bronze badges

1

vote

0answers

22 views

Difference between Dueling DQN and Double DQN?

I have read some articles, but still can not figure out the difference between the Dueling DQN and Double DQN? What exactly is the difference between them? Also, Does Dueling DQN need to be built on ...

reinforcement-learning q-learning dqn deep-network

asked May 31 at 17:46

Edamame

5832 gold badges8 silver badges17 bronze badges

0

votes

0answers

5 views

Deep Q Learning state dimensionality

How important is the dimensionality of each state for Deep Q Learning? I have a set of 15 unique playing cards from a deck of 52 playing cards. A given state is represented by the respective card ...

deep-learning dimensionality-reduction q-learning

asked May 16 at 6:58

Harry Stuart

1063 bronze badges

2

votes

0answers

16 views

Reinforcement Learning using PPO2 in openai gym retro, mario not learning the clear the easy episode

I am training mario game in retro using ppo2 baselines for some time. I have tried level3 and level1 too. But even after full training when I play using saved checkpoints, the mario is not able to ...

reinforcement-learning q-learning openai-gym

asked May 11 at 10:46

Sandeep B

1469 bronze badges

0

votes

0answers

14 views

Reinforcement Learning - Q Learning - Number of Steps to Decrease?

I have an implementation of the Q-Learning algorithm intended to solve the racetrack problem. I have noticed that the initial amount of steps needed to solve the problem is somewhere between 3000-...

machine-learning reinforcement-learning algorithms q-learning

asked May 9 at 1:18

Jerry M.

1084 bronze badges

1

vote

0answers

32 views

Intuition behind the loss function in Deep Q learning?

I'm currently following a tutorial but I got stuck at the deep Q learning model. According to my understanding of neural networks they predict an approximate function for the inputs given with the ...

deep-learning reinforcement-learning loss-function q-learning

asked May 5 at 12:34

anjani

62 bronze badges

2

votes

1answer

22 views

If the set of all possible states changes each time, how can Q-learning “learn” anything?

I found this resource that explains q-learning with a very simple example. Make it a 2D problem, a rectangle instead of a line, and it's still simple. The only difference is that now there are 2 more ...

reinforcement-learning q-learning

asked May 4 at 21:40

Pigna

1333 bronze badges

2

votes

1answer

23 views

Representing state in Q-Learning

I have a fairly simple game in which I wish to use Q-learning to train an agent, but I have some questions regarding state representation. I'm new to RL so bare with me: If you have a game where you ...

reinforcement-learning beginner q-learning

asked May 4 at 9:09

Ross Kohler

111 bronze badge

0

votes

0answers

21 views

Why is the reward fluctuating for Double Q-Learning?

I am trying to implement Double Q-Learning using neural networks from the Keras library. When I first tried Simple DQN, the graph of the reward was fluctuating a lot so, I implemented a Double DQN. ...

deep-learning reinforcement-learning q-learning

asked Apr 20 at 8:19

Gauranga Das

12 bronze badges

0

votes

1answer

63 views

Q-Learning experience replay: how to feed the neural network?

I'm trying to replicate the DQN Atari experiment. Actually my DQN isn't performing well; checking another one's codes, I saw something about experience replay which I don't understand. First, when you ...

python reinforcement-learning q-learning dqn keras-rl

asked Apr 11 at 14:13

Joaquin

12 bronze badges

0

votes

0answers

17 views

Different algorithms categorized in reinforcement learning

(Originally asked at cross validated forum: https://stats.stackexchange.com/questions/401615/different-algorithms-categorized-in-reinforcement-learning) For some time I am going through reinforcement ...

reinforcement-learning q-learning

asked Apr 8 at 20:28

Sandeep B

1469 bronze badges

2

votes

1answer

127 views

DQN fails to find optimal policy

Based on DeepMind publication, I've recreated the environment and I am trying to make the DQN find and converge to an optimal policy. The task of an agent is to learn how to sustainably collect apples ...

reinforcement-learning q-learning dqn convergence deepmind

asked Apr 1 at 1:23

macwiatrak

113 bronze badges

0

votes

0answers

14 views

Is reward accumulated during a play iteration when performing SARSA?

I've been having issue with getting my DQN to converge to a good solution for snake. Regardless of the different types of reward functions I've tried, it seems that the snake is indefinitely going ...

machine-learning deep-learning reinforcement-learning q-learning dqn

asked Mar 29 at 1:44

DevarakondaV

164 bronze badges

0

votes

0answers

20 views

Openai Spaces for a modified environment

I have a 2-dimensional array of normalized data. I am using space = np.array([0,1,...366],[0,0.000001,.....1]) I need to fit this as an observation space in ...

python deep-learning reinforcement-learning q-learning openai-gym

asked Mar 17 at 16:29

Karthik Rajkumar

306 bronze badges

0

votes

0answers

5 views

Q learning transition matrix trouble

First, apologies if this isn't the right forum for my question -- let me know if there's a better place. I'm trying to implement Q learning on a two-dimensional grid world, where the agent has four ...

machine-learning q-learning

asked Jan 30 at 1:14

Elizabeth Han

1012 bronze badges

0

votes

0answers

22 views

Q-Learning where some actions are more difficult to predict than others

I'm trying to train a deep Q network to optimize play in a game. For simplicity, let's say my game only has two actions, A and B. The reward distribution for A is somewhat uniform in the range -1 to ...

machine-learning deep-learning reinforcement-learning q-learning

asked Jan 29 at 20:25

ahbutfore

111 bronze badge

0

votes

1answer

39 views

Why is “next state” kept in RL experience replay?

Following this explanation on what is experience replay (and others), I noticed an experience element is defined as $e_t = (s_t,a_t,r_t,s_{t+1})$ My question is, why do we need the ...

machine-learning reinforcement-learning q-learning policy-gradients

asked Jan 23 at 18:03

Gulzar

1137 bronze badges

0

votes

0answers

9 views

Is every value-iteration off-policy DP is a Q-learning?

Is it true to say that every value-iteration off-policy DP is a Q-learning technique. or there are many more specific definition for it?

q-learning

asked Jan 22 at 19:29

user10296606

3501 silver badge13 bronze badges

1

vote

0answers

23 views

What is the meaning of the Variant Q-learning and To what INPUT and OUTPUT refer? in Abstract of DeepMind DQN paper 2013

-INPUT and OUTPUT OF ATARI DQN: In the abstract paragraph of the DQN work by DeepMind https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf it has written: " We present the first deep learning model to ...

deep-learning reinforcement-learning q-learning dqn deep-network

asked Jan 22 at 10:36

user10296606

3501 silver badge13 bronze badges

2

votes

1answer

185 views

What is the difference between dynamic programming and Q-learning?

What is the difference between the DP-based algorithm and Q-learning?

reinforcement-learning q-learning dynamic-programming

asked Jan 21 at 5:52

user10296606

3501 silver badge13 bronze badges

0

votes

1answer

43 views

How to represent an image as state in a Q-table

I'm trying to do Q-learning with the Atari games using the gym python's package. I want to use the image as the state of my algorithm, but I came up with a doubt: ...

reinforcement-learning q-learning openai-gym

asked Dec 31 '18 at 18:22

Neoares

1185 bronze badges

0

votes

1answer

31 views

How does Q-Learning deal with mixed strategies?

I'm trying to understand how Q-learning deals with games where the optimal policy is a mixed strategy. The Bellman equation says that you should choose $max_a(Q(s,a))$ but this implies a single unique ...

machine-learning reinforcement-learning q-learning

asked Dec 20 '18 at 17:48

Thomas Johnson

3251 gold badge2 silver badges10 bronze badges

0

votes

1answer

54 views

Why not use max(returns) instead of average(returns) in off-policy Monte Carlo control?

As I understand it, in reinforcement learning, off-policy Monte Carlo control is when the state-action value function $Q(s,a)$ is estimated as a weighted average of the observed returns. However, in ...

reinforcement-learning q-learning monte-carlo

asked Dec 20 '18 at 10:52

Meldgaard

11 bronze badge

1

vote

1answer

130 views

Reinforcement learning: negative reward (punish) illegal actions?

If you train an agent using reinforcement learning (with Q-function in this case), should you give a negative reward (punish) if the agent proposes illegal actions for the presented state? I guess ...

machine-learning reinforcement-learning q-learning

asked Dec 2 '18 at 10:04

BigBadMe

3082 silver badges12 bronze badges

1

vote

1answer

620 views

IndexError: index 804 is out of bounds for axis 0 with size 800

i installed a self driving car project from superdatascience site , when i open the map using terminal after a while the map window close up or it closes directly after i maximize the map window and ...

reinforcement-learning python q-learning

asked Nov 5 '18 at 15:05

R.SH

1

vote

1answer

44 views

Will reinforcement learning work if states wont get repeated again?

I am working on a information retrieval model where the user enters a query and the model has to retrieve 3 most relevant FAQ pairs.I am collecting implicit feedback in terms of page clicks etc.What I ...

machine-learning deep-learning reinforcement-learning q-learning dqn

asked Oct 24 '18 at 11:57

Charan Reddy

1061 bronze badge

1

vote

0answers

75 views

Dueling DQN - Calculation of Q-value

I'm trying to implement a Double Dueling DQN on LunarLander and I'm facing an issue as my model is not learning so I'm trying to debug the graph and this leads me to a question regarding the ...

reinforcement-learning q-learning

asked Oct 19 '18 at 15:53

Nicolas M.

1065 bronze badges

1

vote

1answer

62 views

What's going wrong with my Tic Tac Toe Q-Learning Alghoritm?

my Q-learning alghoritm currently choices the "sub optimal" option and not the best one. ...

python q-learning

asked Oct 12 '18 at 20:11

CuriousPanda

63 bronze badges

1

vote

1answer

142 views

What is the immediate reward in value iteration?

Suppose you're given an MDP where rewards are attributed for reaching a state, independently of the action. Then when doing value iteration: $$ V_{i+1} = \max_a \sum_{s'} P_a(s,s') (R_a(s,s') + \...

reinforcement-learning q-learning

asked Oct 8 '18 at 14:38

Dominus

306 bronze badges

4

votes

1answer

1k views

Reinforcement learning: decreasing loss without increasing reward

I'm trying to solve OpenAI Gym's LunarLander-v2. I'm using the Deep Q-Learning algorithm. I have tried various hyperparameters, but I can't get a good score. Generally the loss decreases over many ...

reinforcement-learning q-learning

asked Sep 4 '18 at 12:06

Atuos

1525 bronze badges

1

vote

1answer

2k views

RL Advantage function why A = Q-V instead of A=V-Q?

In RL Course by David Silver - Lecture 7: Policy Gradient Methods, David explains what an Advantage function is, and how it's the difference between Q(s,a) and the V(s) Preliminary, from this post: ...

reinforcement-learning q-learning variance

asked Sep 1 '18 at 3:08

Kari

7985 silver badges26 bronze badges

2

votes

1answer

44 views

How we can have RF-QLearning or SVR-QLearning (Combine these algorithm with a Q-Learning )

How we can have RF-QLearning or SVR-QLearning (Combine these algorithm with a Q-Learning )? I want to replace the DNN section of Qlearning with a RF or SVR but the problem is that there is no clear ...

random-forest reinforcement-learning q-learning dqn svr

asked Aug 31 '18 at 18:01

user10296606

3501 silver badge13 bronze badges

1

vote

0answers

114 views

Deep Q-Learning with large number of actions

I'm using DQN with large number of actions in [0, 10000, step = 1000]. This means I have an action space of size 11 (including 0 and 10000). Action space is still discrete. My problem is that, instead ...

machine-learning deep-learning reinforcement-learning q-learning dqn

asked Aug 16 '18 at 6:04

Krishna Nevase

256 bronze badges

5

votes

1answer

91 views

Representing similar states in reinforcement learning?

Let's say I'd like to design a Q learning algorithm that learns to play poker. The number of different possible States is very large, but a lot are very similar: for example, if the initial state ...

reinforcement-learning q-learning

asked Aug 5 '18 at 21:01

shakedzy

3793 silver badges19 bronze badges

2

votes

1answer

83 views

Calculate Q parameter for Deep Q-Learning applied to videogames

I am working on Deep Q-learning applied to Snake, and I am confused on the methodology. Based on the DeepMind paper on the topic and other sources, the Q-value with the Bellman equation needs to be ...

machine-learning deep-learning reinforcement-learning q-learning

asked Jul 22 '18 at 13:46

maurocomi

547 bronze badges

1

vote

1answer

55 views

Tflearn “nan” weight matrices

I wanted to build a DQN. So I followed this code and watched some videos about the idea of DQN. My Code is this (mine is written in tflearn and his in keras): ...

reinforcement-learning python q-learning

asked Jun 30 '18 at 17:12

Flajt

164 bronze badges

2

votes

1answer

148 views

Dueling DQN what does a' mean?

what does $a'$ mean in the "combining" equation in Dueling DQN? (top of the page 5) $$Q(s,a; \theta, \alpha, \beta) = V(s; \theta, \beta) + \biggl( A(s, a; \theta, \alpha) - \frac{1}{N}\sum_{a'}^{N}A(...

reinforcement-learning q-learning

asked Jun 4 '18 at 9:22

Kari

7985 silver badges26 bronze badges

3

votes

1answer

517 views

Prioritized Experience Replay - why to approximate the Density Function?

I am reading about Prioritized Experience Replay, and can't understand the following: On page 4, every transition can be selected from the table with its own probability. Here is the cumulative ...

reinforcement-learning q-learning

asked May 30 '18 at 23:32

Kari

7985 silver badges26 bronze badges

1

vote

1answer

32 views

Adding a bias makes Q-learning algorithm ineffective

I've been working through the Q-Network learning example in this Arthur Juliani's blog. It's based on the pretty trivia Open Gym Frozen Lake example. It's base implementation get's about 47% success ...

tensorflow reinforcement-learning q-learning

asked May 23 '18 at 2:12

heneryville

1085 bronze badges

3

votes

1answer

41 views

Why does Q-learning use an actor model and critic model?

I'm currently reading Hands on Machine Learning with Scikit-Learn & Tensorflow, and I'm wondering why does Q-learning require an actor model and a critic model to learn? On page 465, it states: ...

deep-learning q-learning

asked May 10 '18 at 10:14

AshleyS

1334 bronze badges

2

votes

1answer

70 views

Experience Replay, must return minibatch back to Memory Bank?

During Experience Replay, we are randomly gathering a minibatch from the Memory bank. We then use the minibatch to correct our NeuralNetwork q-value function approximator. When done, should we return ...

q-learning

asked Apr 25 '18 at 2:13

Kari

7985 silver badges26 bronze badges

1

vote

0answers

127 views

Experience Replay Explain

I have read many blog articles, research papers and watched many youtube videos, but it seems it is hard to find why experience replay is efficient. I know that the experience replay stores (state, ...

deep-learning supervised-learning q-learning dqn

asked Apr 24 '18 at 17:22

Dongyu Kang

61 bronze badge

0

votes

1answer

135 views

Can you interpolate with QLearning or Reinforcement learning in general?

I am currently researching the usages of machine learning paradigms for pathfinding problems. I am currently looking into the reinforcement learning paradigm and I used QLearning for pathfinding. ...

reinforcement-learning q-learning

asked Apr 18 '18 at 8:42

Steven Schenk

31 bronze badge

3

votes

1answer

57 views

What is the optimal value of a Markov Decision process with Single actions at each state?

I am trying to solve some questions about a MRP (i.e. a Markov Decision process with only one possible action at each state). The setup is as follows: There are two states ($a$ and $b$) stepping to $...

machine-learning reinforcement-learning q-learning markov-process

asked Apr 13 '18 at 20:44

BlagBlug1987

161 bronze badge

1

vote

1answer

46 views

Policy gradient on data only, without emulators

It is too costly for my team to emulate the agent (executing the action and assessing the reward), meaning our only option is to learn the optimal policy on our dataset. The good thing is that we have ...

machine-learning data-mining dataset reinforcement-learning q-learning

asked Apr 13 '18 at 10:48

user26616

163 bronze badges

1

vote

1answer

113 views

Reinforcement Learning with static state

Can Q Learning work with a static state for each step? What I mean by that is that the actions do not influence the following state at all. The episodes just iterate over the same data over and over ...

reinforcement-learning q-learning dqn

asked Apr 5 '18 at 14:30

Pit Wegner

61 bronze badge

1

vote

1answer

45 views

How does a Q algorithm consider future rewards?

I am trying to understand the underlying logic of Q learning (deep Q learning to be precise). At the moment I am stuck at the notion of future rewards. To understand the logic, I am reviewing some of ...

machine-learning deep-learning q-learning

asked Apr 5 '18 at 12:14

SercioSoydanov

1083 bronze badges

7

votes

2answers

665 views

Is this a Q-learning algorithm or just brute force?

I have been playing with an algorithm that learns how to play tictactoe. The basic pseudocode is: ...

machine-learning neural-network reinforcement-learning q-learning

asked Mar 10 '18 at 11:03

Ant Kutschera

1415 bronze badges

4

votes

1answer

298 views

Q learning neural network experience replay problem

I am currently trying to create a tic tac toe q learning neural network to introduce me to reinforcement learning, however it didn't work so I decided to try a simpler project requiring a network to ...

machine-learning neural-network q-learning

asked Feb 1 '18 at 15:58

Peter Jamieson

575 bronze badges

2

votes

1answer

1k views

Reinforcement Learning on data only (NO emulators)

My team and I started digging into RL for the purpose of a specific application. We have plenty of data of an agent carrying out suboptimal policies (states and rewards...). It is too costly for us ...

machine-learning dataset algorithms reinforcement-learning q-learning

asked Jan 31 '18 at 20:28

user26616

163 bronze badges

Questions tagged [q-learning]

Related Tags