Newest 'actor-critic' Questions

0 votes

0 answers

8 views

Multi-task reinforcement learning with different action spaces

I'm currently working on a project in which I need apply multi -task reinforcement learning. Over the same state space, each agent aims to do a separate task, but the action spaces of agents are ...

aby

11

asked Apr 26 at 18:06

0 votes

0 answers

34 views

Keras on-policy "Advantage Actor Critic" implementation

understand and implement on-policy "Advantage Actor-Critic" The Keras RL example is straight and simple, it uses The Kersa functional API to create an actor-critic and after each episode ...

keras tensorflow reinforcement-learning actor-critic keras-rl

Osiris

1

asked Mar 9 at 22:18

1 vote

0 answers

115 views

Soft actor-critic reinforcement learning for 100x100 maze environment

I am doing a project which requires a soft actor-critic reinforcement learning agent to learn how to reach a goal in a 100x100 maze environment as the one below: The state space is discrete and only ...

machine-learning deep-learning reinforcement-learning ai actor-critic

reinforcement_learner

13

asked Nov 18, 2021 at 6:52

0 votes

0 answers

23 views

how to in enhance A3C entropy?

I'm trying to implement this A3C code in my custom environment, and I have a basic understanding of the algorithm. The algorithm worked, but it did not give me a good performance. I looked into ...

deep-learning reinforcement-learning pytorch actor-critic

DCnoob

101

asked Oct 6, 2021 at 7:31

0 votes

1 answer

173 views

Actor Network Target Value in A2C Reinforcement Learning

In DQN, we use; $Target = r+\gamma v(s')$ equation to train (fit) our network. It is easy to understand since we use the $Target$ value as the dependent variable like we do in supervised learning. I....

machine-learning reinforcement-learning actor-critic

datatech

53

asked Apr 15, 2021 at 16:43

0 votes

1 answer

419 views

Reinforcement Learning - PPO: Why do so many implementations calculate the returns using the GAE? (Mathematical reason)

There are so many PPO implementations that use GAE and do the following: ...

machine-learning reinforcement-learning mathematics policy-gradients actor-critic

Johannes

1

asked Mar 8, 2021 at 19:39

1 vote

0 answers

113 views

A2C learning very slowly when I try to make it learn on batches as compared to making it learn on each step

I tried this on openai gym environment - LunarLander-v2. I wrote two algorithms with just one difference: Made it learn on each step. Made it learn at the end of each episode. There is a ...

policy-gradients actor-critic

starlord

11

asked Jun 14, 2020 at 12:58

1 vote

0 answers

215 views

Pytorch XLA to solve the spawn problems in a Colab Env

As reference only, here is my code It seems that torch.multiprocessing.set_start_method("spawn") can't be used in an Colab Env. Only 'fork' is allowed. I have ...

python reinforcement-learning pytorch actor-critic

jgauth

31

asked May 22, 2020 at 12:53

1 vote

0 answers

33 views

Action selection in actor-critic algorithm:

I have an action space that is just a list of values given by acts = [i for i in range(10, 100, 10)]. According to pytorch documentary, the loss is calculated as below. Could someone explain to me how ...

reinforcement-learning pytorch actor-critic

EArwa

65

asked Mar 30, 2020 at 12:57

2 votes

0 answers

44 views

Actions taken by agentn/ agent performance not improving

Hi I am trying to develop an rl agent using PPO algorithm. My agent takes an action(CFM) to maintain a state variable called RAT in between 24 to 24.5. I am using PPO algorithm of stable-baselines ...

reinforcement-learning actor-critic keras-rl discounted-reward

cvg

515

asked Jan 21, 2020 at 5:41

1 vote

0 answers

39 views

Rewards are converged but with a lot of variations

I am training a reinforcement learning agent on an episodic task of fixed episode length. I am tracking the training process by plotting the cumulative rewards over an episode. I am using tensorboard ...

reinforcement-learning ai actor-critic

cvg

515

asked Nov 29, 2019 at 10:28

1 vote

1 answer

143 views

Having a reward structure which gives high positive rewards compared to the negative rewards

I am training an RL agent using PPO algorithm for a control problem. The objective of the agent is to maintain temperature in a room. It is an episodic task with episode length of 9 hrs and step size(...

reinforcement-learning dqn ai actor-critic monte-carlo

cvg

515

asked Nov 27, 2019 at 4:39

1 vote

1 answer

50 views

Formulation of a reward structure

I am new to reinforcement learning and experimenting with training of RL agents. I have a doubt about reward formulation, from a given state if a agent takes a good action i give a positive reward, ...

reinforcement-learning ai actor-critic keras-rl discounted-reward

cvg

515

asked Nov 26, 2019 at 10:26

1 vote

0 answers

248 views

Actor Critic Model implementation

I am going to work on a project which requires implementation of A2C model using Tensorflow 2.0. I am new in the Machine Learning field and also in Python. These are topics which I have covered ...

machine-learning deep-learning reinforcement-learning implementation actor-critic

EMT

13

asked Nov 25, 2019 at 0:42

0 votes

1 answer

82 views

How to handle differences between training and deploying of an RL agent

Hi I am training an RL agent for a control problem. The objective of the agent is to maintain temperature in a zone. It is an episodic task with episode length of 10 hrs and actions being taken every ...

reinforcement-learning dqn ai actor-critic monte-carlo

cvg

515

asked Nov 18, 2019 at 7:16

1 vote

1 answer

576 views

Different results every time I train a reinforcement learning agent

I am training an RL agent for a control problem using PPO algorithm. I am using stable-baselines library for it. The objective of an agent is to maintain a temperature of 24 deg in a zone and it ...

deep-learning reinforcement-learning dqn actor-critic monte-carlo

cvg

515

asked Nov 6, 2019 at 11:03

3 votes

1 answer

4k views

Evaluating a trained Reinforcement Learning Agent?

I am new to reinforcement learning agent training. I have read about PPO algorithm and used stable baselines library to train an agent using PPO. So my question here is how do I evaluate a trained RL ...

reinforcement-learning dqn actor-critic monte-carlo keras-rl

cvg

515

asked Oct 30, 2019 at 11:41

3 votes

2 answers

3k views

Agent always takes a same action in DQN - Reinforcement Learning

I have trained an RL agent using DQN algorithm. After 20000 episodes my rewards are converged. Now when I test this agent, the agent is always taking the same action , irrespective of state. I find ...

reinforcement-learning dqn policy-gradients actor-critic

cvg

515

asked Oct 4, 2019 at 15:02

0 votes

1 answer

54 views

Reward is converging but actions taken by trained agent are illogical in reinforcement learning

I am training a reinforcement learning agent using DQN. My state space has 6 variables and the agent can one action which is discretized into 500 actions My reward structure looks like ...

python reinforcement-learning policy-gradients ai actor-critic

cvg

515

asked Oct 3, 2019 at 11:47

1 vote

1 answer

81 views

Why can't Policy Gradient Algorithm be seen as an Actor-Critic Method?

During the equation deducing in policy gradient algorithm(e.g., REINFORCE), we are actually using an expectancy of total reward, which we try to maximize. $$\overline{R_\theta}=E_{\tau\sim\pi_\theta}[...

machine-learning reinforcement-learning q-learning policy-gradients actor-critic

KningTG

11

asked Jul 22, 2019 at 9:54

1 vote

1 answer

820 views

A2C Continuous for Pendulum-v0 working implementation, negation for loss and entropy calculation

very good implementation of A2C continuous for Pendulum-v0 Code has snippet to stop execution when mean of last 10 or 20 is higher than -20 but the results look like: ...

neural-network distribution gaussian openai-gym actor-critic

mLstudent33

552

asked Jun 23, 2019 at 1:25

1 vote

1 answer

435 views

multipying negated gradients by actions for the loss in actor nn of DDPG

In this Udacity project code that I have been combing through line by line to understand the implementation, I have stumbled on a part in class Actor where this ...

policy-gradients actor-critic

mLstudent33

552

asked Mar 24, 2019 at 14:42

1 vote

0 answers

57 views

Reinforcement learning - generating a matrix of continuous values with varying size for test data generation

Currently, I am using RL A3C algorithm for test data generation, where for a set of 30 functions written in C (mostly basic algorithms like Prime number checks, triangle validity, etc.) I try to ...

neural-network reinforcement-learning rnn markov-process actor-critic

Cegy

21

asked Mar 13, 2019 at 1:55

0 votes

2 answers

88 views

Proof subtracting baseline doesn't influence gradient can be used to show no gradient exist at all?

I am using David Silver's course in RL to help me write my thesis. However, I am baffled by the proof given in lecture 7 slide 29: slideshow \begin{align} \mathbb{E}_{\pi_\theta}[\nabla_\theta \log_\...

reinforcement-learning policy-gradients actor-critic

Alex Van de Kleut

11

asked Feb 5, 2019 at 1:21

1 vote

1 answer

112 views

Stability of value function approximation in policy gradients

In DQNs, function approximation of the Q-values is unstable for correlated updates. In policy gradients with a baseline, will the value function of the policy not be plagued by the same correlated ...

neural-network reinforcement-learning policy-gradients actor-critic

Johann

43

asked Oct 16, 2018 at 19:23

3 votes

1 answer

353 views

Time horizon T in policy gradients (actor-critic)

I am currently going through the Berkeley lectures on Reinforcement Learning. Specifically, I am at slide 5 of this lecture. At the bottom of that slide, the gradient of the expected sum of rewards ...

machine-learning deep-learning reinforcement-learning policy-gradients actor-critic

Dummie Variable

76

asked Aug 28, 2018 at 14:56

2 votes

2 answers

609 views

A3C - Turning action probabilities into intensities

I'm experimenting with using an A3C network to learn to play old Atari video games. My network outputs a set of probabilities for each possible action (e.g. left, right, shoot), and I use this ...

machine-learning reinforcement-learning probability actor-critic

Levi Botelho

143

asked Jan 27, 2018 at 12:49

3 votes

1 answer

1k views

How to design two different neural nets for actor and critic RL?

In order to have an actor critic RL model there are two things to be satisfied . Value approximation function should converge to a local minimum $$\sum_s d^{\pi}(s) \sum_a \pi(s,a)[Q^{\pi}(s,a) - ...

reinforcement-learning actor-critic

Shamane Siriwardhana

797

asked Dec 5, 2017 at 11:39

Stack Exchange Network

current community

your communities

more stack exchange communities

Questions tagged [actor-critic]

Multi-task reinforcement learning with different action spaces

Keras on-policy "Advantage Actor Critic" implementation

Soft actor-critic reinforcement learning for 100x100 maze environment

how to in enhance A3C entropy?

Actor Network Target Value in A2C Reinforcement Learning

Reinforcement Learning - PPO: Why do so many implementations calculate the returns using the GAE? (Mathematical reason)

A2C learning very slowly when I try to make it learn on batches as compared to making it learn on each step

Pytorch XLA to solve the spawn problems in a Colab Env

Action selection in actor-critic algorithm:

Actions taken by agentn/ agent performance not improving

Rewards are converged but with a lot of variations

Having a reward structure which gives high positive rewards compared to the negative rewards

Formulation of a reward structure

Actor Critic Model implementation

How to handle differences between training and deploying of an RL agent

Different results every time I train a reinforcement learning agent

Evaluating a trained Reinforcement Learning Agent?

Agent always takes a same action in DQN - Reinforcement Learning

Reward is converging but actions taken by trained agent are illogical in reinforcement learning

Why can't Policy Gradient Algorithm be seen as an Actor-Critic Method?

A2C Continuous for Pendulum-v0 working implementation, negation for loss and entropy calculation

multipying negated gradients by actions for the loss in actor nn of DDPG

Reinforcement learning - generating a matrix of continuous values with varying size for test data generation

Proof subtracting baseline doesn't influence gradient can be used to show no gradient exist at all?

Stability of value function approximation in policy gradients

Time horizon T in policy gradients (actor-critic)

A3C - Turning action probabilities into intensities

How to design two different neural nets for actor and critic RL?

Hot Network Questions

Questions tagged [actor-critic]

Related Tags