Questions tagged [reinforcement-learning]

Area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward.

Filter by
Sorted by
Tagged with
1
vote
1answer
20 views

How to combine two differently scaled, but equally important “running” signals into a reward function?

I asked this question on Artificial Intelligence, but got no answer, so I am moving it here. I have two signals that I want to use to model a reward for a reinforcement learning algorithm. The first ...
0
votes
0answers
10 views

My Deep Q-Learning Network does not learn for OpenAI gym's cartpole problem

I am implementing OpenAI gym's cartpole problem using Deep Q-Learning (DQN). I followed tutorials (video and otherwise) and learned all about it. I implemented a code for myself and I thought it ...
-1
votes
0answers
13 views

Two neural network in tensorflow: TypeError: Fetch argument None has invalid type <class 'NoneType'> [closed]

I have seen similar problems describes typerror problem, but anything I found fit/solve my problem. I have some reinforcement learning task, when I want to use to neural networks to steers two degrees ...
1
vote
1answer
133 views

Tensorflow / Deepmind: how do I take actions from observations for math algorithms related to proofs?

Crossposted from here This question is to ask for directions/suggestions/help on the use of deepmind opensource libraries: DeepMind Lab or TensorFlow in Python. Consider that I'm new to concepts like ...
6
votes
1answer
529 views

Why could my DDQN get significantly worse after beating the game repeatedly?

I've been trying to train a DDQN to play OpenAI Gym's CartPole-v1, but found that although it starts off well and starts getting full score (500) repeatedly (at around 600 episodes in the pic below), ...
2
votes
0answers
315 views

What is significance of Colour-digit MNIST game in paper Learning to Communicate with Deep Multi-Agent Reinforcement Learning?

My question is regarding the paper Learning to Communicate with Deep Multi-Agent Reinforcement Learning. Can anyone explain what is the significance of Colour-digit MNIST game in the paper? I ...
0
votes
1answer
39 views

When should the last action be included in the state in reinforcement learning?

I am having some confusion as to whether the action should be included as part of the state input to an agent in a reinforcement learning setting (state-action pair). As from my observation, this is ...
1
vote
0answers
30 views

What is the meaning of the Variant Q-learning and To what INPUT and OUTPUT refer? in Abstract of DeepMind DQN paper 2013

-INPUT and OUTPUT OF ATARI DQN: In the abstract paragraph of the DQN work by DeepMind it has written: " We present the first deep learning model to successfully learn control policies directly ...
1
vote
1answer
316 views

DQN - how is it possible to train separate outputs for each action?

I'm trying to implement a Deep Q Network, but I'm stuck on how you train a network to predict multiple action-values when you can only collect data on a single action. In the paper it recommends using ...
0
votes
0answers
25 views

Deep Reinforcement Learning with Space Invaders

I want to better understand Deep Reinforcement Learning so I developed the Space Invaders game from scratch with Pygame. I have a fixed number of enemies (10). Instead of defining the states as a ...
1
vote
1answer
217 views

Problem when cherry picking actions - Proximal Policy Optimization

I'm using the implementation of PPO2 in stable-baselines (a fork of OpenAI's baselines) for a RL-problem. My observation space is 9x9x191 and my action space is 144. Given a state, only some actions ...
-2
votes
0answers
23 views

information theory of deep learning , machine learning [closed]

I'm going to talk to a podcast person who works in the following areas:information theory of deep learning What questions do you suggest I ask or recommend to be asked?
5
votes
3answers
3k views

What is a policy in machine learning?

While I was reading the paper "Grounded Action Transformation for Robot Learning in Simulation", I came across the term "policy". Could someone explain to me what that actually is (...
0
votes
0answers
8 views

How do Deep Q networks actually converge

The whole idea of DQNs is converging to our target values, but in supervised learning these values are the unbiased true values In reinforcement learning, the target value is also a prediction, so how ...
1
vote
1answer
59 views

What is the best way Reinforcement learning, RNN or others to predict the best action we have to take to maximize sales?

I have a dataset composed of few features : customerId, actionDay1, SalesDay1, actionDay20, SalesDay20, actionDay30, SalesDay30 action can be : call email face ...
11
votes
2answers
3k views

Prioritized Replay, what does Importance Sampling really do?

I can't understand the purpose of importance-sampling weights (IS) in Prioritized Replay (page 5). A transition is more likely to be sampled from experience replay the larger its "cost" is. ...
0
votes
0answers
27 views

In a double Deep Q network what would happen if we switch the roles of both networks

We normally use the online network for action selection and the target network for evaluation , would there be a difference if we switched the roles? Because in the case Of Double Q learning, we ...
0
votes
0answers
28 views

Reinforcement learning for turn-based AI

For a side project I'm trying to build a (simplified) AI for Heroes Of Might and Magic, using (as a starting point) deep Q-learning. But I'm having trouble to understand how the "state space"...
4
votes
2answers
96 views

Is there a rule of thumb when designing neural network in deep reinforcement learning?

In deep learning, we can assess model's performance with loss function value and improve model's performance with K-fold cross-validation and so on. But how can we design and tune neural network used ...
1
vote
2answers
376 views

Alternative approach for Q-Learning

I have a question related to an alterative Q-Learning approach. I'd like to know if this already exists and I am not aware of it, or it doesn't exist because there are theoretical problems behind it. ...
0
votes
2answers
44 views

Does an RL agent learn during exploitation?

I have started with RL and have some doubts regarding it. Does an RL agent learn during exploitation, or does it only learn during exploration? Is it possible to train a model only using exploitation ...
2
votes
1answer
2k views

IndexError: index 804 is out of bounds for axis 0 with size 800

i installed a self driving car project from superdatascience site , when i open the map using terminal after a while the map window close up or it closes directly after i maximize the map window and ...
0
votes
1answer
30 views

Reinforcement Learning control with known dynamic equation

I know there is model-based reinforcement learning. But all the approaches assume an MDP. If I want to do a feedback control of a system (i. e. control an inverted pendulum) it's quite easy to find ...
2
votes
1answer
120 views

Reinforcement Learning on real time data over a web server

Question: is it possible to implement a reinforcement learning model over a NodeJS server? This server would be receiving binary forms of data (open /close; yes/no) in real time. The objective for ...
1
vote
1answer
89 views

Expected value in Bellman equation

I am reading "reinforcement learning - An introduction" by Sutton and Barto. At pag. 59, there is the Bellman equation for the state-value function $\begin{array}{ll} v_{\pi}(s) &= \mathbb{E}_{...
0
votes
1answer
79 views

Understanding policy gradient theorem - What does it mean to take gradients of reward wrt policy parameters?

I am looking for a little clarity on what the policy gradient theorem means. My confusion lies in the fact that the reward $R$ in reinforcement learning is non-differentiable in the policy parameters. ...
1
vote
0answers
19 views

How can I build a simulation environment that assess different risk policies? [closed]

I work in fin-tech and would like to build some sort of simulation program to assess how different inputs will impact net revenue. For example, if we create new policies based on ML scores, how would ...
2
votes
1answer
117 views

Policy gradient - and auto-differentiation (Pytorch/Tensorflow)

In policy gradient, we have something like this: Is my understanding correct that if I apply log cross-entropy on the last layer, the gradient will be automatically calculated as per formula above?
0
votes
1answer
36 views

Monte Carlo for non-episodic tasks

In Sutton's textbook (Chapter 5) it says "To ensure that well-defined returns are available, here we define Monte Carlo methods only for episodic tasks". Can someone explain what exactly ...
6
votes
1answer
5k views

Q-Learning: Target Network vs Double DQN

I am having a hard time understanding difference between Target Network and Double DQN From this blog: Target Network generates the target-Q values that will be used to compute the loss for every ...
2
votes
2answers
35 views

Idenitity between TD(0) algorithm and Policy Evaluation in Dynamic Programming when alpha is equal to 1

TD(0) algorithm is defined as the iterative update of the following: $$ V(s) \leftarrow V(s) + \alpha({r + \gamma V(s')} - V(s) ) $$ Now, if we assume alpha to be equal to 1, we get the traditional ...
0
votes
2answers
112 views

How to calculate Temperature variable in softmax(boltzmann) exploration

Hi I am developing a reinforcement learning agent for a continous state/discrete action space. I am trying to use boltmzann/softmax exploration as action selection strategy. My action space is of size ...
2
votes
0answers
60 views

Sudden drop of score in the last few episodes

I was following this tutorial about lunar lander and deep Q learning with Tensorflow 2 and I noticed something odd. The problem was actually solved at episode 476 but then the score went from 259.90 ...
0
votes
1answer
42 views

Is there a mistake in Lecture 5 of Stanford CS234 available on youtube?

https://www.youtube.com/watch?v=buptHUzDKcE&list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u&index=5 At 53:45 Professor starts to describe temporal difference for linear value function approximation. ...
1
vote
1answer
87 views

Why not use max(returns) instead of average(returns) in off-policy Monte Carlo control?

As I understand it, in reinforcement learning, off-policy Monte Carlo control is when the state-action value function $Q(s,a)$ is estimated as a weighted average of the observed returns. However, in ...
0
votes
0answers
27 views

How to use Tensorflow in Project Malmo

I've taken a tensorflow for beginner course and I'm wondering if Project Malmo can be use with Tensorflow, just a simple demonstration will do. Specifically, I'm trying to modify tutorial_6 of python-...
0
votes
0answers
25 views

Off-Policy Estimation - Importance Sampling with Negative Rewards

Importance sampling is a common method for calculating off-policy estimates in RL. I have been reading through some of the original documentation (D.G. Horvitz and D.J. Thompson, Powell, M.J. and ...
1
vote
1answer
41 views

Agent Collapse / Overfitting during Training

I'm new to reinforcement learning so please bear with me. I'm training an agent to play ms-Pacman using the actor-critic method. Below are the results of a couple of runs, in both graphs the orange ...
1
vote
1answer
39 views

How to improve tensorflow 2.0 code for policy gradient?

I recreated some code I found online for solving the bandits problem using policy gradient. The example was in tensorflow 1.0 so I recreated it with tensorflow 2.0 using eager execution and gradient ...
10
votes
0answers
194 views

Formal proof of vanilla policy gradient convergence

So I stumbled upon this question, where the author asks for a proof of vanilla policy gradient procedures. The answer provided points to some literature, but the formal proof is nowhere to be included....
1
vote
1answer
31 views

Why does categorical_crossentropy work in when input is not 1-hot encoded?

I'm going through lessons on the REINFORCE algorithm to solve Cartpole/Pong/etc (using AIGym) and every one uses categorical_crossentropy as the loss function. What's confusing me is that ...
1
vote
1answer
12 views

Parameter initialization in a genetic algorithm

I'm using a neural network in a genetic algorithm. The neural network has 4 inputs (values between 0 and 1) and 4 outputs, corresponding to the probabilities of different actions. The neural network ...
1
vote
2answers
42 views

Machine learning goal: given a population of 100,000 students, predict a group of 3,000, and minimize the median grade of that group

In other words, I am looking to predict students that will fail out of school before it happens. The data includes socioeconomic status and other related variables. I have tried an XGB binary ...
0
votes
0answers
19 views

Prioritized Experience Replay - which version is correct?

After reading a lot of stuff, I'm still not sure how to calculate the priorities for Prioritized Experience Replay (PER). Example code taken from here ...
0
votes
0answers
9 views

Wich activation function for DQL

After many research, I still can't find a neat answer about this question: When I found the loss of my state-action pair. I'm only backpropagating that loss true the network and setting all other ...
0
votes
0answers
23 views

Debugging Reinforcement Learning Model

I'm new to RL and I'm attempting to train an RL agent to play MsPacman in PyTorch. I've adapted the code from this tutorial on the PyTorch page for my problem. The DQN has the following architecture: <...
0
votes
1answer
890 views

ValueError: Error when checking input: expected input_1 to have 4 dimensions, but got array with shape (1, 4, 4, 200, 200)

I am working on a dqn agent with a CNN which takes input of 4 images, each grey-scaled image array is of size 80x80. my model structure is like this:- ...
0
votes
1answer
21 views

Policy Gradient not “learning”

I'm attempting to implement the policy gradient taken from the "Hands-On Machine Learning" book by Geron, which can be found here. The notebook uses Tensorflow and I'm attempting to do it with PyTorch....
8
votes
1answer
124 views

Which ML approach to choose for the game AI when rewards are delayed?

Question: Which Machine Learning approach should I choose for the AI of my computer game, where the actions of the AI do not lead to immediate rewards, but delayed rewards instead? About me: I am a ...
0
votes
1answer
208 views

Result of Reinforcement learning

I've started reading some literature about reinforcement learning and I can't understand what is the result of the application of RL. I'll be more specific: let's have a time series problem in ...

1
2 3 4 5
10