Newest 'actor-critic' Questions

0

votes

0answers

3 views

Action selection in actor-critic algorithm:

I have an action space that is just a list of values given by acts = [i for i in range(10, 100, 10)]. According to pytorch documentary, the loss is calculated as below. Could someone explain to me how ...

asked 11 hours ago

EArwa

356 bronze badges

2

votes

0answers

21 views

Actions taken by agentn/ agent performance not improving

Hi I am trying to develop an rl agent using PPO algorithm. My agent takes an action(CFM) to maintain a state variable called RAT in between 24 to 24.5. I am using PPO algorithm of stable-baselines ...

reinforcement-learning actor-critic keras-rl discounted-reward

asked Jan 21 at 5:41

cvg

3971 silver badge12 bronze badges

1

vote

0answers

14 views

Rewards are converged but with a lot of variations

I am training a reinforcement learning agent on an episodic task of fixed episode length. I am tracking the training process by plotting the cumulative rewards over an episode. I am using tensorboard ...

reinforcement-learning ai actor-critic

asked Nov 29 '19 at 10:28

cvg

3971 silver badge12 bronze badges

1

vote

1answer

30 views

Having a reward structure which gives high positive rewards compared to the negative rewards

I am training an RL agent using PPO algorithm for a control problem. The objective of the agent is to maintain temperature in a room. It is an episodic task with episode length of 9 hrs and step size(...

reinforcement-learning dqn ai actor-critic monte-carlo

asked Nov 27 '19 at 4:39

cvg

3971 silver badge12 bronze badges

1

vote

1answer

23 views

Formulation of a reward structure

I am new to reinforcement learning and experimenting with training of RL agents. I have a doubt about reward formulation, from a given state if a agent takes a good action i give a positive reward, ...

reinforcement-learning ai actor-critic keras-rl discounted-reward

asked Nov 26 '19 at 10:26

cvg

3971 silver badge12 bronze badges

0

votes

0answers

10 views

Several dips in accumulated episodic rewardss during training of a reinforcement learning agent

Hi I am training reinforcement learning agents for a control problem using PPO algorithm. I am tracking the accumulated rewards for each episode during the training process. Several times during the ...

reinforcement-learning dqn ai actor-critic keras-rl

asked Nov 25 '19 at 12:31

cvg

3971 silver badge12 bronze badges

1

vote

0answers

72 views

Actor Critic Model implementation

I am going to work on a project which requires implementation of A2C model using Tensorflow 2.0. I am new in the Machine Learning field and also in Python. These are topics which I have covered ...

machine-learning deep-learning reinforcement-learning actor-critic

asked Nov 25 '19 at 0:42

E M Tonoy

111 bronze badge

0

votes

1answer

29 views

How to handle differences between training and deploying of an RL agent

Hi I am training an RL agent for a control problem. The objective of the agent is to maintain temperature in a zone. It is an episodic task with episode length of 10 hrs and actions being taken every ...

reinforcement-learning dqn ai actor-critic monte-carlo

asked Nov 18 '19 at 7:16

cvg

3971 silver badge12 bronze badges

0

votes

0answers

55 views

saving a model during training of an RL agent

I am training an RL agent using PPO2 algorithm. Iam using stable-baselines library. During the training process, my rewards are slowly increasing and stabilizing, but are falling down suddenly. I ...

neural-network deep-learning reinforcement-learning training actor-critic

asked Nov 11 '19 at 14:24

cvg

3971 silver badge12 bronze badges

0

votes

0answers

123 views

Sudden decline in cumulative reward of an reinforcement learning training agent

Hi I am training an RL agent using PPO algorithm of stable baselines library. I have integrated my training with tensorboard to monitor the training process. Over a period of time my training rewards ...

tensorflow reinforcement-learning dqn actor-critic discounted-reward

asked Nov 8 '19 at 11:44

cvg

3971 silver badge12 bronze badges

0

votes

1answer

36 views

Different results every time I train a reinforcement learning agent

I am training an RL agent for a control problem using PPO algorithm. I am using stable-baselines library for it. The objective of an agent is to maintain a temperature of 24 deg in a zone and it ...

deep-learning reinforcement-learning dqn actor-critic monte-carlo

asked Nov 6 '19 at 11:03

cvg

3971 silver badge12 bronze badges

1

vote

1answer

266 views

Evaluating a trained Reinforcement Learning Agent?

I am new to reinforcement learning agent training. I have read about PPO algorithm and used stable baselines library to train an agent using PPO. So my question here is how do I evaluate a trained RL ...

reinforcement-learning dqn actor-critic monte-carlo keras-rl

asked Oct 30 '19 at 11:41

cvg

3971 silver badge12 bronze badges

0

votes

0answers

16 views

RL library for implementing PPO algorithm

I have recently studied about PPO algorithm. I want to train an RL agent using PPO. However I am finding it difficult to implement. So I am planning to use a library which will help me in the ...

tensorflow reinforcement-learning dqn actor-critic keras-rl

asked Oct 21 '19 at 16:07

cvg

3971 silver badge12 bronze badges

1

vote

2answers

268 views

Agent always takes a same action in DQN - Reinforcement Learning

I have trained an RL agent using DQN algorithm. After 20000 episodes my rewards are converged. Now when I test this agent, the agent is always taking the same action , irrespective of state. I find ...

reinforcement-learning dqn policy-gradients actor-critic

asked Oct 4 '19 at 15:02

cvg

3971 silver badge12 bronze badges

0

votes

1answer

28 views

Reward is converging but actions taken by trained agent are illogical in reinforcement learning

I am training a reinforcement learning agent using DQN. My state space has 6 variables and the agent can one action which is discretized into 500 actions My reward structure looks like ...

python reinforcement-learning policy-gradients ai actor-critic

asked Oct 3 '19 at 11:47

cvg

3971 silver badge12 bronze badges

0

votes

0answers

11 views

Actor-critic: Should actor and critic networks have the same size?

I'm using an actor-critic RL approach (DDPG to be specific) and am wondering if there is some rule of thumb that says whether or not the actor NN and critic NN should have the same size, ie, number of ...

neural-network actor-critic

asked Sep 10 '19 at 13:34

CGFoX

1011 bronze badge

1

vote

1answer

20 views

Why can't Policy Gradient Algorithm be seen as an Actor-Critic Method?

During the equation deducing in policy gradient algorithm(e.g., REINFORCE), we are actually using an expectancy of total reward, which we try to maximize. $$\overline{R_\theta}=E_{\tau\sim\pi_\theta}[...

machine-learning reinforcement-learning q-learning policy-gradients actor-critic

asked Jul 22 '19 at 9:54

KningTG

111 bronze badge

0

votes

1answer

257 views

A2C Continuous for Pendulum-v0 working implementation, negation for loss and entropy calculation

very good implementation of A2C continuous for Pendulum-v0 Code has snippet to stop execution when mean of last 10 or 20 is higher than -20 but the results look like: ...

neural-network distribution gaussian openai-gym actor-critic

asked Jun 23 '19 at 1:25

mLstudent33

4381 silver badge14 bronze badges

0

votes

0answers

219 views

Implementation of actor-critic model for MountainCar

I'm trying to build a model for the Mountain Car game, following this Actor-Critic code: https://github.com/nikhilbarhate99/Actor-Critic (However, in this case, it's discrete action space, while it's ...

python reinforcement-learning pytorch actor-critic

asked Apr 5 '19 at 20:42

nolw38

1037 bronze badges

0

votes

0answers

39 views

Speeding up actor critic training

I'm simulating a very simple system, recommendation system, and I am running an actor-critic model to predict what item I should recommend next. The agent is learning and is doing just fine. However, ...

machine-learning reinforcement-learning training pytorch actor-critic

asked Mar 30 '19 at 21:22

Carl Rynegardh

4463 silver badges13 bronze badges

1

vote

1answer

219 views

multipying negated gradients by actions for the loss in actor nn of DDPG

In this Udacity project code that I have been combing through line by line to understand the implementation, I have stumbled on a part in class Actor where this ...

policy-gradients actor-critic

asked Mar 24 '19 at 14:42

mLstudent33

4381 silver badge14 bronze badges

1

vote

0answers

30 views

Reinforcement learning - generating a matrix of continuous values with varying size for test data generation

Currently, I am using RL A3C algorithm for test data generation, where for a set of 30 functions written in C (mostly basic algorithms like Prime number checks, triangle validity, etc.) I try to ...

neural-network reinforcement-learning recurrent-neural-net markov-process actor-critic

asked Mar 13 '19 at 1:55

Cegy

211 bronze badge

0

votes

2answers

40 views

Proof subtracting baseline doesn't influence gradient can be used to show no gradient exist at all?

I am using David Silver's course in RL to help me write my thesis. However, I am baffled by the proof given in lecture 7 slide 29: slideshow \begin{align} \mathbb{E}_{\pi_\theta}[\nabla_\theta \log_\...

reinforcement-learning policy-gradients actor-critic

asked Feb 5 '19 at 1:21

Alex Van de Kleut

112 bronze badges

1

vote

1answer

68 views

Stability of value function approximation in policy gradients

In DQNs, function approximation of the Q-values is unstable for correlated updates. In policy gradients with a baseline, will the value function of the policy not be plagued by the same correlated ...

neural-network reinforcement-learning policy-gradients actor-critic

asked Oct 16 '18 at 19:23

Johann

434 bronze badges

3

votes

1answer

192 views

Time horizon T in policy gradients (actor-critic)

I am currently going through the Berkeley lectures on Reinforcement Learning. Specifically, I am at slide 5 of this lecture. At the bottom of that slide, the gradient of the expected sum of rewards ...

machine-learning deep-learning reinforcement-learning policy-gradients actor-critic

asked Aug 28 '18 at 14:56

Dummie Variable

562 bronze badges

2

votes

2answers

570 views

A3C - Turning action probabilities into intensities

I'm experimenting with using an A3C network to learn to play old Atari video games. My network outputs a set of probabilities for each possible action (e.g. left, right, shoot), and I use this ...

machine-learning reinforcement-learning probability actor-critic

asked Jan 27 '18 at 12:49

Levi Botelho

1233 bronze badges

3

votes

1answer

649 views

How to design two different neural nets for actor and critic RL?

In order to have an actor critic RL model there are two things to be satisfied . Value approximation function should converge to a local minimum $$\sum_s d^{\pi}(s) \sum_a \pi(s,a)[Q^{\pi}(s,a) - ...

reinforcement-learning actor-critic

asked Dec 5 '17 at 11:39

Shamane Siriwardhana

6871 gold badge4 silver badges21 bronze badges

Stack Exchange Network

current community

your communities

more stack exchange communities

Questions tagged [actor-critic]

Action selection in actor-critic algorithm:

Actions taken by agentn/ agent performance not improving

Rewards are converged but with a lot of variations

Having a reward structure which gives high positive rewards compared to the negative rewards

Formulation of a reward structure

Several dips in accumulated episodic rewardss during training of a reinforcement learning agent

Actor Critic Model implementation

How to handle differences between training and deploying of an RL agent

saving a model during training of an RL agent

Sudden decline in cumulative reward of an reinforcement learning training agent

Different results every time I train a reinforcement learning agent

Evaluating a trained Reinforcement Learning Agent?

RL library for implementing PPO algorithm

Agent always takes a same action in DQN - Reinforcement Learning

Reward is converging but actions taken by trained agent are illogical in reinforcement learning

Actor-critic: Should actor and critic networks have the same size?

Why can't Policy Gradient Algorithm be seen as an Actor-Critic Method?

A2C Continuous for Pendulum-v0 working implementation, negation for loss and entropy calculation

Implementation of actor-critic model for MountainCar

Speeding up actor critic training

multipying negated gradients by actions for the loss in actor nn of DDPG

Reinforcement learning - generating a matrix of continuous values with varying size for test data generation

Proof subtracting baseline doesn't influence gradient can be used to show no gradient exist at all?

Stability of value function approximation in policy gradients

Time horizon T in policy gradients (actor-critic)

A3C - Turning action probabilities into intensities

How to design two different neural nets for actor and critic RL?

Hot Network Questions

Questions tagged [actor-critic]

Related Tags