# Questions tagged [q-learning]

A model-free reinforcement learning technique.

109 questions
Filter by
Sorted by
Tagged with
24 views

### Why does the PyTorch tutorial on DQN define state as a difference?

I'm a master's student in EECS working my way towards understanding how DQN  works. I'm working towards solving the CartPole-v0 task in as few iterations as possible. First of all I implemented a ...
26 views

### Cannot solve FrozenLake with QLearning and slippery mode enabled

I'm trying to solve the open-ai gym frozen lake problem with a basic Q learning table and it works quite well with is_slippery=False. But it cannot find a solution ...
29 views

### Definition of the Q* function in reinforcement learning

I'm making my way through Sutton's Introduction to Reinforcement Learning. He gives the definition of the $q_*$ function as follows $$q_*(a) = \mathbf{E}[R_t | A_t = a]$$ where $A_t$ is the action ...
19 views

### Reference implementation of q-learning in Python

I'm a machine learning newbie, trying to learn Q-learning. I read a few texts and I get the general gist, but what I'd really love to see is a simple example of a Q-learning algorithm in Python that I ...
24 views

### Different Initial Q-Values in Q-Learning

When working with Q-Learning, what is the difference between having a Q_0(a) with all values zero, random or optimistic?
18 views

### When should I use normal Q learning over a DQN?

From this article here, it says that using a tabular Q function is less scalable than a deep Q network. I assume that this means that the Q table approach works for some environments, but once they ...
21 views

### My Deep Q-Learning Network does not learn for OpenAI gym's cartpole problem

I am implementing OpenAI gym's cartpole problem using Deep Q-Learning (DQN). I followed tutorials (video and otherwise) and learned all about it. I implemented a code for myself and I thought it ...
32 views

### Deep Q-learning, how to set q-value of non-selected actions?

I am learning Deep Q-learning by applying it to a real world problem. I have been through some tutorials and papers available online but I counldn't figure out the solution for the following problem ...
12 views

### Wich activation function for DQL

After many research, I still can't find a neat answer about this question: When I found the loss of my state-action pair. I'm only backpropagating that loss true the network and setting all other ...
53 views

### Is this a valid stability concern/improvement for DQN/DDQN reinforcement training?

As you all know, DQN or DDQN are known for "unstable training". Let's use the well known "CartPole". The agent has to balance the stick and gets a reward of +1 per frame. You can reach the 195 ...
37 views

### How to save and load a Q-Learning Agent

I know this may sound nooby, but how do I save a Deep Q-Learning agent's progress? I mean when I close at i.e. episode 500 when my agent is trained and I restart (in my case a pygame) my agent is ...
26 views

### First Simple DQN not learning to navigate maze

So I am currently attempting to write my first DQN implementation, where the aim is for the agent to learn to navigate the board from the top left to the bottom right while avoiding the hole right in ...
20 views

### Can we use Q-Learning (or RL in general) for this problem?

Let's say that we have an algorithm that given a dataset point, it runs some analysis on it and returns the results. The algorithm has a user-defined parameter X that affects the run-time of the ...
10 views

### Reward Function for a model-free MDP

I am trying to build a program which has to decide in a completely stochastic environment. So it has to be model-free and Q-learning is suitable for that. I just have one problem, my rewards are not ...
8 views

### For off policy reinforcement learning how different can the current policy be from the policies which generated the data

Say we have two policies and we use one to generate data with. We now want to use this data to optimize the second policy (the two policies are defined with the same input and output space but with ...
6 views

### Reward engineering to replace single terminal reward (exponential utility of terminal wealth)

My goal is to use reinforcement learning to train the agent (the trader) to maximize the exponential utility of his P&L (profit and loss) at a terminal time T. Therefore the natural formulation of ...
21 views

### How are n dimensional vectors state vectors represented in Q Learning?

Using this code: ...
69 views

### Reward(t) vs. Reward(t+1) ? Reinforcement Learning, Q-learning

In "Reinforcement Learning, An Introduction; Richard S. Sutton and Andrew G. Barto" is written on page 70: 3.1 The Agent–Environment Interface At each time step t, the agent receives some ...
140 views

### Index tensor must have same dimensions as input tensor

I am trying to train a DQN to do optimal energy scheduling. Each state comes as a vector of 4 variables (represented by floats) saved in the replay memory as a state tensor, each action is an integer ...
91 views

### Q-learning, state transition, immediate rewards (trading logic)

I've been thinking about how to correctly calculate rewards for several weeks now. Here is a grid example: ...
16 views

### find the parameter of model with Q learning

I have a question with regard to Q learning. I am a beginner in Q learning. Every example that I saw is related to the environment that the goal is assigned to a place. (like cliff walk that we know ...