In Sutton and Barto Book, under section 10.3. Average Reward: A New Problem Setting for Continuing Tasks, what is the use of subtracting reward and the average reward? What are they trying to solve here, I understood what an average reward is, but failed to understand why it is being subtracted from the reward at each time step. (shown in the images below)Average Reward

where average reward is given by enter image description here


1 Answer 1


Section 10.3 presents a nondiscounted continual learning setting.

If we use the rewards as is (without subtracting the empirical mean reward at time step t), then the return and value function can grow without bound.

On the other hand, the differential return $G_t$ as you screenshotted above converges to zero in the limit of $t$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.