What does this value function of a parameterized policy mean? and is it related to TRPO in RL?

Iv been watching the RL lectures on youtube from Stanford. In episode 9 – policy gradients 2 the teacher Emma Brunskill says we are going to learn about how to make safer policy gradient steps by adjusting (not sure how yet) the policy gradient estimate we compute. The teacher says the equation for the value function of the policy parameterized by $$\theta$$ is the following.

$$V(\theta) = E_{\pi\theta} [\sum_{t=0}^{\infty} \gamma^{t} R(s_t,a_t)]$$

where $$s_0 \sim \mu(\pi)$$ the start state distribution, $$a_t \sim \pi(a_t|s_t)$$ the policy, $$s_{t+1} \sim P(s_{t+1}|s_t,a_t)$$ the transition model of a MDP that terminates or is episodic, $$R(s_t,a_t)$$ is the reward model evaluated for the state and action expected at time step t.

Then the teacher says we can write the value function of another policy like the following. I get lost here.

$$V(\tilde{\theta}) = V(\theta) + E_{\pi\tilde{\theta}}[\sum_{t=0}^{\infty} \gamma^{t} A_{\pi}(s_t,a_t)]$$

then she re-expresses that expectation as.

$$V(\tilde{\theta}) = V(\theta) + \sum_{s}\mu(s)\sum_{a}\pi(a|s) A_{\pi}(s,a)$$

Where $$V(\theta)$$ is the value function of the policy parameterized by $$\theta$$ and $$\tilde{\theta}$$ is a different set of parameters than $$\theta$$. I don’t understand why? Im looking for an explanation of what this equation is used for? Or why it makes sense to represent the value function of a policy using the value function and advantage function of another policy.

Does this mean that $$V(\tilde{\theta})$$ is expected to be different from $$V(\theta)$$ by the the advantage function of $$\pi\theta$$ averaged by the distribution of states and actions seen under $$\pi\tilde{\theta}$$ Im having a hard time wrapping my mind around that. I think it means that if the new policy chooses actions differently than the current policy it will expect to get advantage that the current policy expects to get from taking those actions. (and additionally weighted by the discounted distribution of states seen undere the new policy)

I haven’t read the cource testbook Reinforcement Learning: an introduction by Sutton & Barto. If you have and this was part of the book and you understand why this makes sense please let me know.