# Advantage Function - Variance Reduction

When explaining advantage function, it is usually claimed that using a baseline reduces the variance. I have not found any specific reference to justify this.

Is this an application of control variates or something similar?

Could anyone provide some reference or formal justification for the variance reduction?

• no answers at all? I can provide some additional background if necessary Jan 4 '17 at 19:23
• That would help! Could you elaborate on the idea behind those claims? Jan 5 '17 at 7:36

As you mention it can be viewed as a control covariate addition [4] which is used to reduce variance in Monte Carlo estimations. A good choice for that function is to use the usual value function ($V(s)$) which reduces the variance of your estimate.