Questions tagged [reinforcement-learning]

Area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward.

Filter by
Sorted by
Tagged with
1
vote
1answer
47 views

What is the best way Reinforcement learning, RNN or others to predict the best action we have to take to maximize sales?

I have a dataset composed of few features : customerId, actionDay1, SalesDay1, actionDay20, SalesDay20, actionDay30, SalesDay30 action can be : call email face ...
4
votes
2answers
83 views

Is there a rule of thumb when designing neural network in deep reinforcement learning?

In deep learning, we can assess model's performance with loss function value and improve model's performance with K-fold cross-validation and so on. But how can we design and tune neural network used ...
1
vote
2answers
118 views

Alternative approach for Q-Learning

I have a question related to an alterative Q-Learning approach. I'd like to know if this already exists and I am not aware of it, or it doesn't exist because there are theoretical problems behind it. ...
2
votes
0answers
37 views
+50

Actor-critic architecture: How is the policy updated?

I am going through the ddpg baseline code to try and gain an intuitive understanding of how the actor and critic networks function. DDPG has two components: the actor which is the deterministic ...
0
votes
2answers
35 views

Does an RL agent learn during exploitation?

I have started with RL and have some doubts regarding it. Does an RL agent learn during exploitation, or does it only learn during exploration? Is it possible to train a model only using exploitation ...
1
vote
1answer
1k views

IndexError: index 804 is out of bounds for axis 0 with size 800

i installed a self driving car project from superdatascience site , when i open the map using terminal after a while the map window close up or it closes directly after i maximize the map window and ...
1
vote
2answers
456 views

Does reinforcement learning only work on grid world?

Does reinforcement learning always need a grid world problem to be applied to? Can anyone give me any other example of how reinforcement learning can be applied to something which does not have a ...
0
votes
0answers
15 views

Assumptions on discounted long-term loss

The infinite horizon discounted long-term loss is defined as: $$ f(\theta) = \mathbb{E}_{\tau \sim \mathbb{P}(.|\theta)}\left[\sum_{t=1}^{\infty}{\gamma^t l_m(s_t,a_t)}\right]$$ where $(s_t,a_t) \in ...
7
votes
2answers
113 views

Representing similar states in reinforcement learning?

Let's say I'd like to design a Q learning algorithm that learns to play poker. The number of different possible States is very large, but a lot are very similar: for example, if the initial state ...
0
votes
0answers
8 views

exploitation vs. exploration: upper-confidence-bound vs. epsilon-greedy

I am looking into some different ways for doing exploitation vs. exploration (e.g. multi-arm bandit problem). There are approaches like upper_confidence_bound and epsilon-greedy. I am wondering what ...
0
votes
0answers
21 views

reinforcement learning: PPO vs. DDPG vs. TRPO - difference and intuition

I know there is a lot of blog talk about the PPO, DDPG and TRPO, but I am wondering would it be possible to explain the differences of these methods in layman's term? What's the intuition behind them ...
1
vote
0answers
9 views

Rewards are converged but with a lot of variations

I am training a reinforcement learning agent on an episodic task of fixed episode length. I am tracking the training process by plotting the cumulative rewards over an episode. I am using tensorboard ...
1
vote
0answers
10 views

Deep Q Learning - training slows down significantly

I'm trying to build a deep Q network to play snake. I designed the game so that the window is 600 by 600 and the snake's head moves 30 pixels each tick. I implemented the DQN algorithm with memory ...
3
votes
1answer
91 views

Choosing a right algorithm for template-based text generation

I am doing a text generation project -- the task is to basically represent the statistical data in a readable way. The way I decided to go about this is template-based: each data type has a template ...
1
vote
1answer
23 views

Having a reward structure which gives high positive rewards compared to the negative rewards

I am training an RL agent using PPO algorithm for a control problem. The objective of the agent is to maintain temperature in a room. It is an episodic task with episode length of 9 hrs and step size(...
0
votes
1answer
20 views

Reinforcement Learning in NLP for chatbots

Is anyone aware of any successful implementation of reinforcement learning for NLP. I am looking to for chatbots which can learn automatically. Tried searching internet but found very few articles ...
1
vote
0answers
9 views

Autonomous learning - chatbots

My chatbots need to be trained when we get new data or feedbacks from users. Can someone provides ways how these chatbots can learn on themselves and become intelligent day by day? Some of techniques ...
0
votes
0answers
12 views

using reinforcement learning for classification

this is a purely conceptual query and in case the moderators feel it needs to be asked elsewhere, i would be happy to move it there. We do a lot of work in text classification and a senior ...
0
votes
2answers
33 views

Proof subtracting baseline doesn't influence gradient can be used to show no gradient exist at all?

I am using David Silver's course in RL to help me write my thesis. However, I am baffled by the proof given in lecture 7 slide 29: slideshow \begin{align} \mathbb{E}_{\pi_\theta}[\nabla_\theta \log_\...
0
votes
1answer
19 views

Reinforcement Learning control with known dynamic equation

I know there is model-based reinforcement learning. But all the approaches assume an MDP. If I want to do a feedback control of a system (i. e. control an inverted pendulum) it's quite easy to find ...
2
votes
1answer
93 views

Reinforcement Learning on real time data over a web server

Question: is it possible to implement a reinforcement learning model over a NodeJS server? This server would be receiving binary forms of data (open /close; yes/no) in real time. The objective for ...
0
votes
0answers
9 views

DNN loss gets smaller but accuracy stays the same

I am learning a DeepNN to choose between three decisions in a simulation. Therefore, I can run the simulation as often as I want and can generate as many samples as I want. Based on this tutorial (...
1
vote
1answer
14 views

Formulation of a reward structure

I am new to reinforcement learning and experimenting with training of RL agents. I have a doubt about reward formulation, from a given state if a agent takes a good action i give a positive reward, ...
1
vote
1answer
36 views

Expected value in Bellman equation

I am reading "reinforcement learning - An introduction" by Sutton and Barto. At pag. 59, there is the Bellman equation for the state-value function $\begin{array}{ll} v_{\pi}(s) &= \mathbb{E}_{...
0
votes
0answers
8 views

Several dips in accumulated episodic rewardss during training of a reinforcement learning agent

Hi I am training reinforcement learning agents for a control problem using PPO algorithm. I am tracking the accumulated rewards for each episode during the training process. Several times during the ...
0
votes
0answers
13 views

Actor Critic Model implementation

I am going to work on a project which requires implementation of A2C model using Tensorflow 2.0. I am new in the Machine Learning field and also in Python. These are topics which I have covered ...
0
votes
1answer
55 views

Understanding policy gradient theorem - What does it mean to take gradients of reward wrt policy parameters?

I am looking for a little clarity on what the policy gradient theorem means. My confusion lies in the fact that the reward $R$ in reinforcement learning is non-differentiable in the policy parameters. ...
0
votes
0answers
6 views

two PPO implentations

I have uploaded here my PPO implentation from scratch: https://github.com/MakisKans/Reinforcement_Learning/tree/master/PPO In the PPO.py you can find two functions : learn and train. The former is ...
0
votes
0answers
15 views

What reinforcement learning algorithm to choose for self-driving car

I have a car that has three sensors at the front. Using these sensors only I want to let it learn to drive on a track. I'm new to reinforcement learning, but I was thinking about using the Q-learning ...
1
vote
1answer
17 views

Training a model that has both 2D and 1D features using a CNN

I'm looking to pre-train a model for an RL agent but I'm having some trouble figuring some stuff out. Dataset: Minerl MineRLNavigateDense-v0 The observation space includes : 2D screen input (64,64) ...
1
vote
1answer
80 views

Policy gradient - and auto-differentiation (Pytorch/Tensorflow)

In policy gradient, we have something like this: Is my understanding correct that if I apply log cross-entropy on the last layer, the gradient will be automatically calculated as per formula above?
0
votes
0answers
24 views

Applying Reinforcement Learning in the following scenario

I'm working on a scenario/environment where I have a simulation that provides an arrangement or results of the simulation that has data in a format of samples in vectors(x,y,z,N). Let's say it maps ...
0
votes
1answer
18 views

How to handle differences between training and deploying of an RL agent

Hi I am training an RL agent for a control problem. The objective of the agent is to maintain temperature in a zone. It is an episodic task with episode length of 10 hrs and actions being taken every ...
5
votes
1answer
234 views

how to choose between discounted reward and average reward

how to select between average reward and discounted reward? And when average reward is more effective in comparison with discounter reward and when vice versa is correct? -Is is possible to use both ...
1
vote
1answer
75 views

How to avoid overfitting in Reinforcement Learning

I have implemented a RL model based on Deep Q-Learning for learning how to play a 2D game, like the ones in the OpenAI Gym. For testing the model, unlike most people, I have chosen to evaluate its ...
1
vote
2answers
102 views

Agent always takes a same action in DQN - Reinforcement Learning

I have trained an RL agent using DQN algorithm. After 20000 episodes my rewards are converged. Now when I test this agent, the agent is always taking the same action , irrespective of state. I find ...
1
vote
0answers
19 views

How can I increase the speed and performance of my implementation of an AI for Reversi?

I made an AI for Reversi, aka Othello (8×8), like Alpha Zero, using this book. This book is written in Japanese. The source code of the AI I implemented can be found in this Github repository. There ...
0
votes
0answers
14 views

Prerequisites for reinforcement learning for DTR

I want to learn how RL is used for Dynamic Treatment Regimes. What are some must-read articles or papers? Also, I am relatively new to RL and am currently reading Sutton's book on the topic. Lets say ...
0
votes
0answers
10 views

saving a model during training of an RL agent

I am training an RL agent using PPO2 algorithm. Iam using stable-baselines library. During the training process, my rewards are slowly increasing and stabilizing, but are falling down suddenly. I ...
1
vote
0answers
18 views

DQN vs PG - when to use which?

I'd like to understand when using PG methods is more adequate than using DQN methods. Just to give a bit of background: I am currently using both APEX and R2D2 for my projects. Both work very well in ...
0
votes
0answers
30 views

Sudden decline in cumulative reward of an reinforcement learning training agent

Hi I am training an RL agent using PPO algorithm of stable baselines library. I have integrated my training with tensorboard to monitor the training process. Over a period of time my training rewards ...
0
votes
1answer
24 views

Different results every time I train a reinforcement learning agent

I am training an RL agent for a control problem using PPO algorithm. I am using stable-baselines library for it. The objective of an agent is to maintain a temperature of 24 deg in a zone and it ...
0
votes
0answers
10 views

Gym Cartpole not solving with Cross Entropy Method?

Cross Entropy Method is considered as one of the simplest optimization algorithm which can be used for training an agent. I tried to train an agent to solve gym's cartpole environment and I have used ...
0
votes
0answers
6 views

Why does my Deep Q Model only take a single action?

I don't know if this is the proper place to ask code-based questions on but I've been struggling with this issue for a while. Basically I am training a Deep Q Model using Keras and Google Colab (for ...
0
votes
0answers
15 views

What's the input for the cost function?

I'm trying to implement deep Q-learning, but I do not know what to put into the cost function. My net has 8 scalar inputs, 4 scalar outputs (from 0-1) and no hidden layers. To calculate the cost I ...
0
votes
0answers
9 views

How to use deep reinforcement learning to learn how to play Checkers?

I am a student new in reinforcement learning and I'm trying to implement an AI able to play Checkers. I want to implement a deep learning solution. However, I am confused on how to do that. I ...
1
vote
1answer
69 views

Why not use max(returns) instead of average(returns) in off-policy Monte Carlo control?

As I understand it, in reinforcement learning, off-policy Monte Carlo control is when the state-action value function $Q(s,a)$ is estimated as a weighted average of the observed returns. However, in ...
0
votes
0answers
2 views

Best DDPG use case when having sensor historical data and virtual environment of a real world system

I have a historical data (from real sensors) which have enough knowledge of the actions and states needed for the use of reinforcement learning and a modeled virtual environment of a real system (...
0
votes
0answers
6 views

Learning using DDPG with pyhton solely using historical data

I have a lengthy timeseries datasets which contains several variables (from sensors etc) to be classified as actions or states. Providing they are successfully done, I want to learn a control policy ...
1
vote
1answer
35 views

Evaluating a trained Reinforcement Learning Agent?

I am new to reinforcement learning agent training. I have read about PPO algorithm and used stable baselines library to train an agent using PPO. So my question here is how do I evaluate a trained RL ...