3
$\begingroup$

In Deep Mind's Rainbow paper, how come A3C algorithm be so slow? twice slower than DDQN... Was it trained on a single actor? :D

It's on page 1 of the paper

enter image description here

Wasn't A3C supposed to be something a lot more powerful?

For example as splendid as this, taken from here:

enter image description here

$\endgroup$

1 Answer 1

1
$\begingroup$

Keep in mind that the DQN based method have replay memory that is sampled and used to optimize the model at every time step while A3C is only optimized when the episode finishes. So for 4000 timesteps, and 1000 timestep per episode, A3C optimized 4 times while DQN could have optimized 3000+ times depending on the implementation. This may explain why given the same number of timesteps, DQN is learning faster.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.