Data Science

Questions tagged [q-learning]

Ask Question

A model-free reinforcement learning technique.

129 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
1
vote
0answers
10 views

How to create transition probability (state) for q-learning algorithm designed to control traffic light system using python?

I am trying to create a q learning algorithm to control traffic light systems. I am representing the state with a matrix : ...
0
votes
1answer
39 views

Deep Q-learning

I am working on the DDQN algorithm which is given in the following paper. I am facing a problem with the Q value. The author calculate Q value by this Q(s, a; θ , α, β) = V(s; θ , β) + A(s, a; θ , α). ...
0
votes
0answers
26 views

Q value is estimated under state V value and action A value for DDQN

How Q value is estimated under state V value and action A value. Given the below DDQN algorithm, the deep network is divided into two parts on the end layer, including state value function V(s) which ...
0
votes
1answer
30 views

Understanding DQN Algorithm

Im studying the deep q learning algorithm. You can see it in the picture here: DQN I have a few questions about the deep q learning algorithm. What do they mean with row 14: ...
0
votes
0answers
30 views

Am I using this neural network in a wrong way?

I'm trying to solve a RL problem; the Contextual Bandit problem using Deep Q Learning. My data is all simulated. I have this environment: ...
1
vote
0answers
20 views

How to construct Q-table for complex, large and dynamic spaces in python?

I am trying to construct a Q-table. I have state space and action space. State space consists of large number of complex and dynamic number of elements, but discrete. Theoretically, I understood ...
2
votes
1answer
47 views

Efficient way to tackle card games with many q-table states?

I'm currently in the process of developing an AI for a popular card game here in Germany (called "Schafkopf"). Obviously, one could try to find a perfect strategy with the help of some game ...
0
votes
0answers
20 views

snake reinfocement learning does not learn

I’m new to reinf learning, and I’m trying this code, but the snake always goes right without learning, I can’t find the bug, could you help me? THE SNAKE CLASS ...
0
votes
0answers
15 views

Random exploration of an arbitrary agent (or dumb agent) in Deep RL

I'm trying to implement an agent in RL and I was wondering if it's pertinent to take random actions in DeepRL. I see that in traditional Q-learning that sometimes we take random actions to encourage ...
0
votes
0answers
22 views

Deep-DQN implementation that works with (state,action) pairs

I'm moderately new to RL(Reinforcement learning) and trying to solve a problem by using a deep Q learning agent (trying a bunch of algorithms) and I don't want to implement my own agent (I would ...
0
votes
0answers
10 views

In reinforcement learning, continue update weights after training?

Assuming an rl agent that has to be trained, and then you can mainain same weights over episodes: in a single episode, you have to firtly backup weights, udpate continuosly weights in the episodes, ...
1
vote
1answer
39 views

In DQN, why not use target network to predict current state Q values?

In DQN, why not use target network to predict current state Q values, and not only next state q values? In doing a basic dq learning algorithm with nn from scratch, with replay memory, and minibatch ...
0
votes
2answers
34 views

Zero Padding in Convolutional neural network

We use Convolutional neural network because it by design learns features that generalize over spatial location , so when using conv operation it reduce image size and that what we hope to have so we ...
0
votes
0answers
16 views

Convolutional neural network with Deep Q learning in games

What does "the average magnitude of maximal action value output by the network" tell us? I mean if we plot this graph, is it good to start as low value and then increase until it goes in a ...
3
votes
1answer
61 views

Exploration in Q learning: Epsilon greedy vs Exploration function

I am trying to understand how to make sure that our agent explores the state space enough before exploiting what it knows. I am aware that we use epsilon-greedy approach with a decaying epsilon to ...
0
votes
0answers
147 views

Reinforcement Learning in continuing tasks (non-episodic)

I'm looking for an algorithm for Deep Reinforcement Learning in non-episodic or continuing tasks. To be explicit, I'm looking for an algorithm that allows the agent to learn online without separate ...
1
vote
0answers
25 views

DQN CartPole-v1 neural network doesn't optimize

I'm doing my first dnq algorithm, I'm trying to build a dnq agent, and neural network from scratch, but it seems that neural network doesn't optimize, I did 2 hidden layers, with ReLU, and the output ...
2
votes
0answers
26 views

"Learning" algorithm to use when future depends on past events (MDP property not met)

There are around 5 different retirement plans available in my country. People can pick from them freely. I would like to create a solution that would try to predict the best plan(s) given a particular ...
0
votes
0answers
18 views

How to use binary numbers and plain floats in the same neural network input for reinforcement learning?

I'm modeling a system whose configuration can be represented by a binary array ([1,0,0,1] or [0,1,0,0], for example), and the agent can move on a 2D space (thus having 3 DoFs), and the action the ...
1
vote
1answer
35 views

On what principle did Google's DeepMind learn to walk?

I just saw this video on Youtube. On what principle did Google's DeepMind learn to walk? Was it Q-Learning or a Genetic Algorithm or Policy Gradient?
2
votes
1answer
41 views

Dimensionality of the target for DQN agent training

From what I understand, a DQN agent has as many outputs as there are actions (for each state). If we consider a scalar state with 4 actions, that would mean that the DQN would have a 4 dimensional ...
0
votes
0answers
21 views

What are the differences between DQN DDQN and Dueling Q network?

What are the differences between DQN DDQN and Dueling Q network? And how they expressed in the agent's behaviour? (For example in an Open AI Gym environment)
1
vote
0answers
59 views

Deep Q-learning in non-episodic tasks

I want to use Deep Q-learning (specifically DDQL by Hasselt et al. 2015, but it is the same principle) in a non-episodic task (continuing). I know that it is possible to use Q-learning in continuing ...
1
vote
0answers
42 views

DQN DDQN and 3DQN differences?

I'm doing a course on reinforcement learning, and one of our tasks is to implement an agent on the Lunar lander continuous V2 environment from openAI gym. In order to solve the continuous problem, I ...
1
vote
0answers
9 views

Implementing DQN - Pylessons.com article - Queries [closed]

I have followed the below link to implement DQN Algorithm https://pylessons.com/CartPole-reinforcement-learning/ Can someone explain me: Why do we need to have else condition in act function during ...
2
votes
1answer
82 views

Does convergence equal learning in Deep Q-learning?

In my current research project I'm using the Deep Q-learning algorithm. The setup is as follows: I'm training the model (using Deep Q-learning) on a static dataset made up of experiences extracted ...
0
votes
0answers
16 views

How can my loss be stable while the gradient keeps growing?

I have been working on an Offline/Batch Reinforcement Learning problem where I am using a BCQ-DDQN model as a Q-table. The model input is a state of 8 dimensions, and the output is a vector of Q-...
0
votes
1answer
40 views

How to report results of RL with high variance?

I run Q-learning and SARSA algortihms on the same problem but the results fluctuate heavily and when I draw them, there is no smooth graph. How should I repost the results? I run algorithms for 500 ...
0
votes
0answers
13 views

Tensorflow deep Q neural network wont learn with more than 1 hidden layer

I've been using TensorFlow in Python with a very simple deep Q neural network (NN) to play a very simple game. The game is in an array [0, 0, 0, 0] and by picking one of the positions 0, 1, 2, 3, the ...
2
votes
0answers
70 views

Offline/Batch Reinforcement Learning: Doubly Robust Off-policy Estimator takes huge values

Context: My team and I are working on a RL problem for a specific application. We have data collected from user interactions (states, actions, etc.). It is too costly for us to emulate agents. We ...
0
votes
0answers
64 views

Cannot solve FrozenLake with QLearning and slippery mode enabled

I'm trying to solve the open-ai gym frozen lake problem with a basic Q learning table and it works quite well with is_slippery=False. But it cannot find a solution ...
1
vote
1answer
97 views

Definition of the Q* function in reinforcement learning

I'm making my way through Sutton's Introduction to Reinforcement Learning. He gives the definition of the $q_*$ function as follows $$ q_*(a) = \mathbf{E}[R_t | A_t = a] $$ where $A_t$ is the action ...
2
votes
0answers
24 views

Reference implementation of q-learning in Python

I'm a machine learning newbie, trying to learn Q-learning. I read a few texts and I get the general gist, but what I'd really love to see is a simple example of a Q-learning algorithm in Python that I ...
0
votes
1answer
42 views

Different Initial Q-Values in Q-Learning

When working with Q-Learning, what is the difference between having a Q_0(a) with all values zero, random or optimistic?
0
votes
1answer
56 views

When should I use normal Q learning over a DQN?

From this article here, it says that using a tabular Q function is less scalable than a deep Q network. I assume that this means that the Q table approach works for some environments, but once they ...
0
votes
1answer
79 views

Deep Q-learning, how to set q-value of non-selected actions?

I am learning Deep Q-learning by applying it to a real world problem. I have been through some tutorials and papers available online but I counldn't figure out the solution for the following problem ...
1
vote
0answers
13 views

Wich activation function for DQL

After many research, I still can't find a neat answer about this question: When I found the loss of my state-action pair. I'm only backpropagating that loss true the network and setting all other ...
0
votes
1answer
138 views

Is this a valid stability concern/improvement for DQN/DDQN reinforcement training?

As you all know, DQN or DDQN are known for "unstable training". Let's use the well known "CartPole". The agent has to balance the stick and gets a reward of +1 per frame. You can reach the 195 ...
0
votes
1answer
139 views

How to save and load a Q-Learning Agent

I know this may sound nooby, but how do I save a Deep Q-Learning agent's progress? I mean when I close at i.e. episode 500 when my agent is trained and I restart (in my case a pygame) my agent is ...
1
vote
0answers
91 views

Reward(t) vs. Reward(t+1) ? Reinforcement Learning, Q-learning

In "Reinforcement Learning, An Introduction; Richard S. Sutton and Andrew G. Barto" is written on page 70: 3.1 The Agent–Environment Interface At each time step t, the agent receives some ...
1
vote
0answers
95 views

Q-learning, state transition, immediate rewards (trading logic)

I've been thinking about how to correctly calculate rewards for several weeks now. Here is a grid example: ...
2
votes
1answer
36 views

Markov Decision Process representation

I'm attempting to model a simple process using a Markov Decision Process. Let $A$ be a set of $3$ actions : $ A \in \{b,s\}$. $T(s,a,s')$ represents the probability of if in state $s$ , take action $...
1
vote
0answers
20 views

help understanding deep Q learning algorithm from deep mind paper

I'm studying this paper https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf, and I have a question about proposed algorithm(it appears on page 7 of the paper): The problem I see is ...
1
vote
1answer
40 views

How is the target_f updated in the Keras solution to the Deep Q-learning Cartpole/Gym algorithm?

There's a popular solution to the CartPole game using Keras and Deep Q-Learning: https://keon.github.io/deep-q-learning/ But there's a line of code that's confusing, this same question has been asked ...
2
votes
2answers
126 views

Deep Q Network gives same Q values and doesn't improve

I'm trying to build a deep Q network to play snake. I've run into an issue where the agent doesn't learn and its performance at the end of the training cycle is to repeatedly kill itself. After a bit ...
2
votes
1answer
381 views

Would Deep Q Learning work for a finite horizon problem?

I want to apply Deep Q Learning to a problem, which has a clear finite horizon definition, like: $$V(s) = \mathbb{E}[r_1 + r_2]$$ Since the horizon is finite, I do not use reward discounting. My ...
2
votes
0answers
52 views

Incentivizing curiosity in a sparse reward environment

I'm quite new to reinforcement learning, but have been exploring different kinds of architectures (DQN, dueling DQN, actor critic, etc.) and evaluating their ability to solve certain problems. The ...
1
vote
0answers
20 views

Deep Q Learning - training slows down significantly

I'm trying to build a deep Q network to play snake. I designed the game so that the window is 600 by 600 and the snake's head moves 30 pixels each tick. I implemented the DQN algorithm with memory ...
1
vote
0answers
236 views

Reducing the training time of an RL agent

I am trying to develop an rl agent using DQN algorithm.During training, the agent interacts with environment which is a simulated one.Each episode takes around 10 mins to run. This way if want my ...
1
vote
1answer
114 views

How to formulate reward of an rl agent with two objectives

I have started learning reinforcement learning and trying to apply it for my use case. I am developing an rl agent which can maintain temperature at a particular value, and minimize the energy ...

1
2 3

Your privacy

By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy.