What is the difference between State Value function and Return for Markov Reward process ( MRP)?

Question

I have been going through Stanford Lecture on RL. I see in MRP that Return function is same as State Value function. Both are getting expected sum of reward keeping discount factor in mind.

Although in lecture, Professor said that for stochastic process, these values will be different. I am not able to understand this. Any help is appreciated.

Answer 1

They are indeed same in deterministic MRPs. For stochastic MRPs from your lecture note formalization it's apparent that the return is a function of random variables $r_t, r_{t+1}, r_{t+2}, ...$, while on the other hand state value function is the conditional expectation of the same random variables.