I have been going through Stanford Lecture on RL. I see in MRP that Return function is same as State Value function. Both are getting expected sum of reward keeping discount factor in mind.

Although in lecture, Professor said that for stochastic process, these values will be different. I am not able to understand this. Any help is appreciated.

enter image description here


1 Answer 1


They are indeed same in deterministic MRPs. For stochastic MRPs from your lecture note formalization it's apparent that the return is a function of random variables $r_t, r_{t+1}, r_{t+2}, ...$, while on the other hand state value function is the conditional expectation of the same random variables.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.