2
$\begingroup$

The Generalized Advantage Estimator seems very effective with algorithms like PPO in reinforcement learning. But most of the A3C algorithms I have seen use Averaged Advantage estimates. Is there any reason?

$\endgroup$

1 Answer 1

1
$\begingroup$

Sure can:

https://github.com/ikostrikov/pytorch-a3c

The reason is the GAE paper was released after A3C, so the paper implementation, which is what most people will look at, doesn't use GAE.

Relevant lines here: https://github.com/ikostrikov/pytorch-a3c/blob/master/train.py#L95-L98

$\endgroup$
1
  • $\begingroup$ Hi, sometimes when the rollout length of an A3C agent is low GAEs tends to make the training process unstable. $\endgroup$ Commented Dec 16, 2018 at 22:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.