I've been trying to add GAE to my A2C implementation for a while now, but I' can't quite seem to grok how it works.
My understanding of it, is that it reduces the variance of the advantage estimation function by kind of 'averaging out' (or generalising) the advantages based off the values in the rollout.
I tried to run through the maths on my own, and in the end I just had one advantage for the whole rollout, is this right? Normally, we'd have one advantage for each timestep in the rollout.
Can anyone provide an explanation on the intuition of GAE?