Questions tagged [reinforcement-learning]

Area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward.

Filter by
Sorted by
Tagged with
0
votes
0answers
22 views

Applying Reinforcement Learning in the following scenario

I'm working on a scenario/environment where I have a simulation that provides an arrangement or results of the simulation that has data in a format of samples in vectors(x,y,z,N). Let's say it maps ...
0
votes
1answer
12 views

How to handle differences between training and deploying of an RL agent

Hi I am training an RL agent for a control problem. The objective of the agent is to maintain temperature in a zone. It is an episodic task with episode length of 10 hrs and actions being taken every ...
0
votes
0answers
13 views

Reinforcement Learning [on hold]

I am currently looking for some industrial projects in Reinforcement Learning, I want to monitor/schedule server jobs, so please if anyone with reinforcement learning experience could help me in that....
1
vote
0answers
18 views

How can I increase the speed and performance of my implementation of an AI for Reversi?

I made an AI for Reversi, aka Othello (8×8), like Alpha Zero, using this book. This book is written in Japanese. The source code of the AI I implemented can be found in this Github repository. There ...
0
votes
0answers
13 views

Prerequisites for reinforcement learning for DTR

I want to learn how RL is used for Dynamic Treatment Regimes. What are some must-read articles or papers? Also, I am relatively new to RL and am currently reading Sutton's book on the topic. Lets say ...
0
votes
0answers
8 views

saving a model during training of an RL agent

I am training an RL agent using PPO2 algorithm. Iam using stable-baselines library. During the training process, my rewards are slowly increasing and stabilizing, but are falling down suddenly. I ...
1
vote
0answers
12 views

DQN vs PG - when to use which?

I'd like to understand when using PG methods is more adequate than using DQN methods. Just to give a bit of background: I am currently using both APEX and R2D2 for my projects. Both work very well in ...
0
votes
0answers
19 views

Sudden decline in cumulative reward of an reinforcement learning training agent

Hi I am training an RL agent using PPO algorithm of stable baselines library. I have integrated my training with tensorboard to monitor the training process. Over a period of time my training rewards ...
0
votes
1answer
24 views

Different results every time I train a reinforcement learning agent

I am training an RL agent for a control problem using PPO algorithm. I am using stable-baselines library for it. The objective of an agent is to maintain a temperature of 24 deg in a zone and it ...
0
votes
0answers
10 views

Gym Cartpole not solving with Cross Entropy Method?

Cross Entropy Method is considered as one of the simplest optimization algorithm which can be used for training an agent. I tried to train an agent to solve gym's cartpole environment and I have used ...
0
votes
0answers
6 views

Why does my Deep Q Model only take a single action?

I don't know if this is the proper place to ask code-based questions on but I've been struggling with this issue for a while. Basically I am training a Deep Q Model using Keras and Google Colab (for ...
0
votes
0answers
8 views

How to use deep reinforcement learning to learn how to play Checkers?

I am a student new in reinforcement learning and I'm trying to implement an AI able to play Checkers. I want to implement a deep learning solution. However, I am confused on how to do that. I ...
0
votes
0answers
15 views

What's the input for the cost function?

I'm trying to implement deep Q-learning, but I do not know what to put into the cost function. My net has 8 scalar inputs, 4 scalar outputs (from 0-1) and no hidden layers. To calculate the cost I ...
0
votes
0answers
2 views

Best DDPG use case when having sensor historical data and virtual environment of a real world system

I have a historical data (from real sensors) which have enough knowledge of the actions and states needed for the use of reinforcement learning and a modeled virtual environment of a real system (...
0
votes
0answers
6 views

Learning using DDPG with pyhton solely using historical data

I have a lengthy timeseries datasets which contains several variables (from sensors etc) to be classified as actions or states. Providing they are successfully done, I want to learn a control policy ...
1
vote
1answer
26 views

Evaluating a trained Reinforcement Learning Agent?

I am new to reinforcement learning agent training. I have read about PPO algorithm and used stable baselines library to train an agent using PPO. So my question here is how do I evaluate a trained RL ...
1
vote
0answers
32 views

Transitioning from Math PhD to ML research [closed]

I am currently a Math PhD about to defend in January. I work in a field in functional analysis that uses a lot of measure theory (but no stats). I have been considering transitioning careers since I ...
1
vote
2answers
19 views

Deep Q network disregards input, giving identical output no matter the input state

I've creating a very simple game, The board is an array of size 6. 0 is "empty cell" , 5 is "Goal", 8 is "player location" [8 0 0 5 0 0] for example means the agent needs to move 2 "right" to win. ...
1
vote
0answers
12 views

What are regret bounds?

I searched for the term and it appears in a few articles but it is used without explanation. The only explanation I could find is in a PhD thesis: "Regret bounds are the common thread in the analysis ...
0
votes
0answers
10 views

RL library for implementing PPO algorithm

I have recently studied about PPO algorithm. I want to train an RL agent using PPO. However I am finding it difficult to implement. So I am planning to use a library which will help me in the ...
2
votes
1answer
27 views

transform a supervised neural network to reinforcement learning?

I have a functional LSTM model that works with an acceptable performance. How can I now convert this supervised model to a reinforcement learning model for improving the performane? Is there any ...
-1
votes
0answers
20 views

Did I understand deep Q leaning right? (Implementation)

Gday guys, so I tried to implement my own enviroment and agent in order to fully understand DQNs. The enviroment is a dungeon with five states. actionspace = 2 statespace = 5 !!!Action a0 is ...
0
votes
1answer
26 views

Policy Gradient with continuous action space

How to apply reinforce/policy-gradient algorithms for continuous action space. I have learnt that one of the advantages of policy gradients is , it is applicable for continuous action space. One way I ...
0
votes
0answers
10 views

Major drop in DeepQNetwork agent after certain number of iteration

I've built a game with an agent moving around trying to collect gold (+10 reward) and it dies when it hits a wall (-100, terminal condition) and -1 for any other step that is not gold nor wall. My ...
1
vote
1answer
37 views

Deep reinforcement learning on changing data sizes

I have a game that I want to build a model that will learn to play the game. Yet, the environment output is two lists that represent the location and number of soldiers of the user and Opponent. The ...
1
vote
2answers
92 views

Agent always takes a same action in DQN - Reinforcement Learning

I have trained an RL agent using DQN algorithm. After 20000 episodes my rewards are converged. Now when I test this agent, the agent is always taking the same action , irrespective of state. I find ...
1
vote
0answers
17 views

Policy Gradient custom loss function not working

I was experimenting with my policy gradient reinforcement learning algorithm, and I was wondering if I could use a similar method to the supervised cross-entropy. So, instead of using existing labels, ...
0
votes
1answer
23 views

Reward is converging but actions taken by trained agent are illogical in reinforcement learning

I am training a reinforcement learning agent using DQN. My state space has 6 variables and the agent can one action which is discretized into 500 actions My reward structure looks like ...
1
vote
0answers
18 views

How to teach algorithm to mimic paths in a certain enviroment

I have a set of scenarios which represent the movement of a car in a certain environment containing some obstacles. So for each scenario I have the position of the car (x,y,t) and a description of the ...
1
vote
1answer
46 views

How to avoid overfitting in Reinforcement Learning

I have implemented a RL model based on Deep Q-Learning for learning how to play a 2D game, like the ones in the OpenAI Gym. For testing the model, unlike most people, I have chosen to evaluate its ...
0
votes
0answers
12 views

How to calculate Temperature variable in softmax(boltzmann) exploration

Hi I am developing a reinforcement learning agent for a continous state/discrete action space. I am trying to use boltmzann/softmax exploration as action selection strategy. My action space is of size ...
0
votes
0answers
11 views

Python Framework for Evolutionary Strategies Deep Reinforcement Learning

I am looking to experiment with deep reinforcement learning. In particular training with evolutionary strategies. Which python framework is most suited to this? From what I can gather so far: ...
1
vote
0answers
11 views

temperature variable in boltzmmann-exploration in reinforcement learning

I have been using epsilon greedy action selection strategy and recently have come across boltzmann(softmax) action selection strategy. One thing I am not clear about boltzmann exploration is the ...
1
vote
0answers
38 views

boltzmann-exploration(softmax exploration) in reinforcement learning

I have started learning reinforcement learning and as a part of it I am exploring the action selection strategies available. I am comparing epsilon-greedy vs boltzmann exploration(softmax exploration)....
0
votes
0answers
33 views

Having Discrete and Continuous actions in Reinforcement Learning

I would like to design an agent that takes three actions in an environment. Two of which are continuous while the third is discrete. Is there an algorithm in RL that can accomplish this?
0
votes
1answer
13 views

Value function and Q-value

I am new to Reinforcement Learning, and I am having trouble understanding the difference between Value function and Q-value Here is my current understanding: Q-value is just a value for a particular ...
0
votes
0answers
37 views

Reinforcement Learning on a dynamic action space

I wish to train a reinforcement learning model to create a binary space partitioning (BSP) tree for any given unordered set of unique 2D points. In fact, I wish to create multiple RL models each ...
1
vote
1answer
29 views

How do the policy gradient's cost function and gradients work?

I am not a math expert but have a basic understanding of linear algebra, calculus and probability and I understand the math behind back propagation. Currently I am trying to learn about policy ...
0
votes
0answers
23 views

Reducing the training time of an RL agent

I am trying to develop an rl agent using DQN algorithm.During training, the agent interacts with environment which is a simulated one.Each episode takes around 10 mins to run. This way if want my ...
0
votes
0answers
50 views

Prioritized Experience Replay - for whole episodes

I want to use Prioritized Experience Replay for whole episodes, instead for single transitions. What's the best way to define the priorities - as episodes can be of different lengths? Personally I ...
1
vote
1answer
22 views

How to formulate reward of an rl agent with two objectives

I have started learning reinforcement learning and trying to apply it for my use case. I am developing an rl agent which can maintain temperature at a particular value, and minimize the energy ...
1
vote
1answer
55 views

Policy gradient vs cost function

I was working with continuous system RL and obviously stumbled across this Policy Gradient. I want to know is this something like cost function for RL? It kinda gives that impression considering we ...
1
vote
0answers
18 views

Deep Q Learning - How is the ground truth obtained?

I am new to reinforcement learning so I apologize for the wrong use of terms, if any. In SARSA, the value of a state-action pair is updated after the robot takes an action following its internal ...
0
votes
0answers
27 views

How can Reinforcement Learning Work in Trading?

I have recently become interested in Reinforcement Learning, mostly as a result of Alpha Zero’s success in the chess (and my own enthusiasm for the game). While I understand the utility of RL in board ...
2
votes
1answer
25 views

Modelling of an environment that is stochastic in nature

I have started learning reinforcement learning and have few doubts regarding model based and model free methods. Is it possible to model an environment that is stochastic in nature? Is it because ...
1
vote
0answers
24 views

Keras high loss and high accuracy in gk bot with reinforcement learning?

I'm making goal-keeper bot in haxball game. It worked well when i trained less but i worked worse when i trained more. Last reinforcement state: 5160 episode - 4171281 steps - 0.05 epsilon: Last fit ...
0
votes
0answers
31 views

How can I create a loss function in Keras for policy gradient that deals with the fact episodes have varying length?

I am implementing policy gradient to solve the OpenAI CartPole game. I have a loss function that goes as follows: ...
0
votes
0answers
18 views

Why residual gradient algorithm is stable to converge to a suboptimal?

naive residual-algorithm discribed in book RLAI Chapter 11.5 by Sutton and Barto is worked as: $$ \begin{aligned} \mathbf{w}_{t+...
0
votes
0answers
11 views

Hints for large, variable-size action space

I am making an environment using OpenAI gym for Diplomacy, and making an AI for it. In Diplomacy, a player has many units, and each unit has a number of moves available to it. Therefore, the player'...
1
vote
1answer
21 views

Is DQN limited to working with only image frames?

I have few questions about Deep Q Network. Does DQN only accept image frames as input? I have never hear (read) a paperwork where it doesn't use image frames. If the first is a No, then does image ...